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Evaluation 



The Need for Study on These Topics 

Why is it important to study the selection and evaluation 
of teachers? The answer is found in three ideas: 

(a) independently considered, the selection of teachers 
and the evaluation of teachers each has an impact on the 
learning of school children; (b) selection and evaluation 
are related to each other in such a manner that the im- 
pact on learning of children is increased when the two 
processes interact effectively, and (c) current events in 
education indicate that there are forces affecting both 
of these processes. 

In what way does selection of teachers affect learning of 
school children? The "fit** of the teacher and the assign- 
ment determines the types of activities in the classroom 
and the eventual learning of children. However this 
**fit** is determined by much more than a final decision to 
place a teacher in a given setting, since the final choice 
is dependent on what has occurred in prior stages. The 
total selection process includes: 

9 determination of need , with an adequate specification 
which includes precise information on the quantity 
and quality of tasks required as well as the student 
learning expected; 

• recruitment , which provides a sufficient supply of 
people capable of performing required tasks ; 

• selection , which is designed to insure a high proba- 
bility that learning will occur in the classroom; and 

• placement , which attempts to **fit** the person chosen 
to the peculiarities of a given assignment. 

Every stage of the process affects the final placement of 
the teacher in the classroom, and the teacher in turn 
affects the environment of the school children. 

How does evaluation affect classroom learning? Basically, 
evaluation provides information for making corrections if 
goals are not being accomplished. For example, if adequate 
classroom learning is not occurring and/or if designated 
procedures are not being followed, evaluation of teachers 
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does two things: (a) it indicates that something is 

wrong with either the procedures or the output, and 
(b) it provides diagnostic information which indicates 
the potential source of the difficulty. Therefore, 
evaluation provides the basic information necessary to 
correct problems. Usually, correction occurs through 
supervision and inservice training. If classroom learning 
goals are being met, evaluation verifies this and rein- 
forces continuation of similar classroom activities. 

Under these circumstances, evaluation is the means for 
maintaining a given level of procedures and learning 
outputs. 

In order to improve the operation of an organization, 
some people place heavy emphasis on the selection of 
personnel; others stress the training of people once they 
are on the job, inasmuch as jobs change rapidly and 
people adapt to these changes. There is evidence that 
some teachers respond better to on-the-job training 
than others (Short: 1968), and this suggests the need 
to select people who are most capable of performing the 
initial tasks required and who are most likely to learn 
from the supervision and inservice training which will be 
provided . 

Since evaluation is the basis for this supervision and 
training, there is an obvious interaction of selection 
and evaluation. Learning of students is somewhat improved 
by good teacher selection procedures; likewise, evalua- 
tion of teachers improves classroom instruction; how- 
ever, classroom learning is facilitated most by selection 
of teachers who respond favorably to evaluation and 
supervision procedures. In addition, this interaction 
between selection and evaluation suggests the desirability 
of using the same criteria for selection that will even- 
tually be used for evaluation, and of providing informa- 
tion from the evaluation process that will help to correct 
errors in the selection process. 

What events are presently occurring on the educational 
scene that make the study of selection and evaluation 
procedures important? First, the supply-demand ratio is 
higher than in previous years. When the supply of 
teachers is low, a district must increase its activities 
in the recruitment phase of selection; whereas when the 
supply is greater than the demand, the selection and place- 
ment phases must be emphasized. Nationally, the present 
supply-demand ratio warrants increased attention to 
selection and placement. 

Second, there is much discussion among educators and 
citizens' groups of education's responsibility to the 
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public for acconiplisliment of goals. Sometimes this 
pressure for ’’accountability^^ is simply a guise for 
reducing taxes for school support; but often it repre- 
sents a sincere desire to know what tax money is pur- 
chasing and to improve the educational system so that 
children learn more effectively. To know v^7hether certain 
goals are being accomplished and what each teacher is 
contributing to the learning of children, it is necessary 
to evaluate teachers. To evaluate teachers fairly, it is 
necessary to use more effective eva], nation procedures 
than those predominantly used in the past. Hence, the 
current increased interest in evaluation of teachers. 

Intended Audience 

Tliis report is intended for use by people engaged in 
selecting and evaluating teachers. Because of the 
diversity of the audience that will be interested in this 
report, it is unlikely that this study will serve all 
people equally well. However, the report has been organ- 
ized and written with the intent of facilitating its use 
by a wide spectrum of readers. 

Organization of Repo rt 

The report begins with a summary chapter. Very little 
technical background is needed to read this initial chap- 
ter, and almost anyone interested in the selection or 
evaluation of teachers should have little difficulty with 
the concepts. The first chapter includes an overview of 
findings from current practices in education , governmental 
agencies, and industry as well as findings in the research 
literature. However, its primary intent is to provide 
applications for and implications of the research findings 
so that the ideas can be quickly implemented in school 
situations. Some readers may want to read only the first 
chapter, or only those sections of the first chapter which 
concern them. 

Chapter 2 is included for those who are interested in the 
procedures used to collect and analyze the information 
used in the report. It is intentionally brief, and persons 
interested :.n additional specific information regarding 
procedures are encouraged to write the Project Director. 

Chapters 3 and 4 furnish detailed background information 
regarding selection and evaluation processes. Some of 
the conclusions stated in Chapter 1 are elaborated and 
documented , 



vii 



7 



Chapter 5 is a summary of what the research team con- 
siders to be significant trends or themes in the informa- 
tion acquired. This chapter is an attempt to view the 
information from a somewhat different perspective in 
order to determine how the ideas can be unified. The 
terminology of this final chapter may not be as familiar 
to some readers as that of the first chapter, and the 
concepts are somewhat more complex than those in 
Chapter 1. However, readers who are interested primarily 
in applicational material may still find Chapter 5 of 
interest, since it does provide a different viewpoint 
to what is happening in selection and evaluation of 
teachers , Those who find Chapters 3 and 4 stimulating 
are also likely to find Chapter 5 of interest. 

Several appendixes are provided in addition to a biblio- 
graphy. The practitioner, the Board of Education member, 
and the student of administration should all find the 
bibliography beneficial as a source of information 
regarding problems of selection and evaluation of 
teachers . 

Topic The ideas within the chapters are organized around topics. 

Outline and the outline of these topics can be found in the Table 

of Contents. Generally, the material regarding selec- 
tion is separated from the material regarding evaluation, 
although the development of the ideas is somewhat 
parallel . 

A number of ways of organizing the material were consid- 
ered. For example, because of the parallelism of ideas, 
the processes of selection and evaluation might have been 
consolidated rather than discussed separately. This would 
have increased the reader's awareness (at a rather general 
level) of the similarity of the two processes, but would 
have made it more difficult to discuss specific differ- 
ences. Or, had the ideas been organized around sources, 
practices in business, practices in governmental 
agencies, practices in education, research in behavioral 
sciences, research in education, such an organization 
would have clarified how many ideas were acquired from 
various sources, but would have offered very few addi- 
tional advantages. 

The present organization allows a reader to quickly find 
ideas of interest to him. For example, if a new princi- 
pal is interested in information regarding recruiting 
teachers, he will find a summary of this information in 
Chapter 1. If he desires more detailed information, he 
can look at a single location in Chapter 3. The organiza- 
tion, then, is designed to facilitate the search for infor- 
mation by the practitioner who is likely to be using the 
report . 
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Format 



The format of the report is also designed for easy use 
by the practitioner . In addition to the subdivision's 
within each chapter, the format includes the following: 

Key ideas are enclosed in blocks , writ te.n in 
italics. 

At the end of the key idea statement within 
the blocks, sources of information are identi- 
fied according to whether they initially came 
from: (a) research literature, (b) vie.\Nrpoints 

expressed in the literature, or (c) practices 
currently occurring in either industry., educa- 
tion, or governmental agencies. For example 
(research: Jones, 1941), or (practice: industry). 

Key ideas are categorized according to whether 
they are suggestions, notes, cautions, or 
conclusions , 

Arrows (1^) or dots (•) are used to draw the 
reader’s attention to listings , headings , or sub- 
stantive ideas. 

Within the body of the text, references are made 
to other sources of information by citing the 
author and the data of the publication, e.g., 
(Brown, 1962), The complete information on that 
source can then be found by referring to the 
bibliography beginning on page 216. 
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SUMMARY 



The general purpose of this study was to acquire and 
synthesize research and development information 
regarding selection and evaluation of teachers in 
such a manner that this information could be used by 
practicing administrators, teachers, and board of 
education members to improve school personnel practices . 

There was a systematic search of the literature in the 
behavioral sciences on selection and evaluation pro- 
cedures, Questionnaires and interviews were used to 
collect information about current practices of school 
systems , businesses and industries , and governmental 
agencies. Questionnaires were sent to 26 of the 40 
largest school districts in the U, S. plus three sub- 
urbs of each of these districts ; questionnaires from 
67 districts were analyzed and interviews were con- 
ducted with personnel in 31 of the 67 districts. In 
addition, questionnaires were sent to 95 businesses 
and 19 governmental agencies; questionnaires were 
analyzed from 41 organizations and interviews were 
conducted with 16 of them. School districts and bus- 
inesses represented all regions of the U. S. 

Information from the written research and’ development 
reports , questionnaires , and interviews were organized 
so that contrasts and comparisons could be made, A 
preliminary draft of this report was presented to an 
Advisory Committee (consisting of practitioners and 
researchers) for criticism . Following the critique , 
modifications were made to assist in the application 
of the ideas to current personnel problems in schools. 

Results of the study indicate that school districts 
vary considerably in their practices, many of them 
are not implementing recent research findings, and in 
general they are not using practices which industry 
considers routine. 

There seems to be agreement that: 

• The reasons for establishing comprehensive 
and systematic teacher selection and evalua- 
tion procedures are to improve learning con- 
ditions for children and to facilitate 
administrative decisions . 

• Personnel procedures are most effective when 
they are cooperatively planned by teachers 
and administrators. 
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® All personnel who are involved in select lim 
and evaluation should understand the purposes , 
the nature of procedures to be followed, and 
roles of the various people involved. 

0 Training in the specialized tasks of selection 
and evaluation increases the effectiveness of 
personnel involved in these tasks , 

• In order for selection and evaluation procedures 
to continue to serve their purposes, they must 
be checked periodically for sources of errors. 

Six major trends or ideas seem to permeate both researcli 
and practice in the selection and evaluation of 
teachers : 

0 Examination, by a systems analysis approach, of 
the functional relationships among selection, 
evaluation , and supervision of teachers prevents 
isolation of any one function, 

0 The current complex models of selection and 
evaluation of teachers , incorporating criteria 
that are multiple , dynamic , and specific to the 
situation are more realistic and useful than 
prior models. 

0 '^Management by objectives" can clarify organi- 
zational goals and show how these goals can be 
subdivided for action by work units and indi- 
viduals . Accountability is facilitated by 
these procedures . 

0 External evaluation can s timulate self evaluation, 
which leads to more continual diagnosis of indi- 
vidual behavior . 

0 Concurrent analysis of the results of produc t 
and process in a specific situation can be used 
to obtain feedback for correction of individual 
problems, 

0 The manner in which information is managed, i,e., 
the acquisition, processing, and analysis and 
interpretation of information , will affect the 
quality of decisions made in the selection and 
evaluation of teachers , 

Obviously, these general trends or ideas are inter- 
related; incorporation of all six in a personnel 
system is not only possible but desirable. 
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Chapter 1 



SUMMARY OF IMPLICATIONS AND APPLICATIONS 
OF RESEARCH AND PRACTICE IN SELECTION 
AND EVALUATION OF TEACHERS 



The estimated number of teachers in U. S. classrooms 
during the spring of 1970 was just slightly less than two 
million; the number of teachers hired for the 1970-71 year 
is estimated to be approximately 240,000. For each 
teacher hired, a number of others are considered and 
rejected; each teacher in the U. S. is evaluated by some- 
one, either formally or informally , and decisions are 
made based on each evaluation. 

CONCLUSION 



Literally millions of decisions concerned with 
selection and evaluation of teachers are being 
made annually in the [/. S.; each decision has 
a "potential impact on school children^ 
(research: NBA Research Report^ 1969-R14) 



In order for teachers to make a maximum contribution to 
the education of children, they should be: (a) selected 

and placed in situations best suited for their talents, 
and (b) evaluated and provided with feedback relating to 
specified goals . 

Suppose you had the responsibility of selecting and evalu- 
ating teachers in a school district. How would you approach 
the task? How would you make certain that you knew how 
many teachers to hire and how to recruit them effectively? 
Whom would you involve in determining what is effective 
teaching? How would you organize for collecting and analy- 
zing information? What strategies would you use for making 
decisions? What procedures would you use for making alter- 
ations in the system which presently exists? 

This chapter examines these types of questions by reviewing 
some of the resuJts of research and practice and suggesting 
some implications and applications of these results . The 
general intent is to assist practicing administrators, 
teachers, and board of education members to apply research 
and development results to the processes of selection and 
evaluation of teachers. 
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Need for Educational administrators are in an advantageous and 

Information unique position to develop procedures which will use 

the talents of teachers and other professional personnel 
maximally. Therefore, they should be following the best 
practices available for selecting and placing teachers 
in situations best suited to their talents and for eval- 
uating the performance of teachers with relation to 
specified goals. Yet , teacher selection and evaluation 
practices of many districts fall far short of the inno- 
vative practices of business , industry , governmental 
agencies, and some of the better school systems. 

One reason for this lag is unawareness of the total scope 
of promising practices that can be implemented within the 
constraints of local situations. In addition, education- 
al administrators are often unaware of the extensive 
research in the behavioral sciences , business , and govern- 
mental agencies that is directly pertinent to the selec- 
tion and evaluation of teachers. Modern personnel 
practices account for much of the adaptability and rapid 
progress of business organization, and these procedures 
should be applied to education where practicable and 
appropriate. 

Synthesis Another reason for lack of innovation in the selection 

Desirable and evaluation of teachers in some districts is the 

absence of a synthesis of information which relates the 
two processes so they can be examined simultaneously. 

Since validation of the selection process depends on 
data collected in the evaluation process, the standards 
of teaching success in both should be compatible. Also, 
the decision to place a teacher in a given position is 
a part of the selection process which involves specific 
evaluation and supervisory procedures. For new teachers, 
the probationary period is an extension of the selection 
process, during which additional data (collected for 
evaluation) are available for making a decision to 
retain, reassign, or release the teacher . All of these 
interrelationships indicate that it is desirable to 
study the two processes concurrently. 

In this chapter, results from research and current prac- 
tices are made available in a format that will assist 
practitioners in changing procedures. The chapter is 
organized in two parts: selection of teachers, and 

evaluation of teachers. Each part has an abstract, sub- 
divisions presenting the significance or purpose of the 
item, findings or conclusions , and implications or 
suggestions for application. 
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ABSTRACT: 



SELECTION OF TEACHERS 



Unless selection procedures are well organised 
and executed^ school districts will not attract 
the number and type of applicants desired or 
they will hire ineffective teachers. Therefore, 
the purpose of a selection procedure is to 
develop adequate ways to: 

0 determine specific teacher needs in terms 
of number of positions and the general 
nature of the positions 

0 determine specific teacher attributes for 
each position 

0 recruit desirable applicants by locating 
sources of supply and devising ways to 
attract applications 

0 collect and process data from written 
documents, tests, and interviews 

0 select and place the best- fitted applicants 

0 establish controls over the entire process 
to determine any sources of errors. 

■Even with a large number of applicants, one cannot 
choose the best teachers without a good selection 
program i however, the possibility of selecting 
good teachers is greatly reduced when a district is 
unable to attract an adequate supply of applicants , 
since the output of a selection program (the tea- 
cher hired) is no better than the input (the appli- 
cant pool) . 

The information collected via written documents, 
tests, and interviews must be reliable measures 
of those attributes Judged to be most important 
as predictors of successful teaching in specific 
situations. Further, this information must be 
analysed and interpreted correctly if good choices 
are to be made. In order for a selection process 
to continue to function smoothly, it must be 
checked periodically for sources of errors. 
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Determination of Teacher Needs 



Purpose Prior to initiating a recruitment campaign for new teachers 

or any other selection activities, it is necessary to 
specify how many and what type of teachers will be 
needed. The purpose for doing this is to indicate 
resources needed to accomplish goals of the school system. 

Presently, teacher needs are generally determined differ- 
ently from personnel needs in industry. Industrial needs 
are based on an annual plan of operation which specifies 
goals and resources needed to accomplish these goals; 
teacher needs are usually based on some arbitrary pupil- 
teacher ratio or extrapolation from prior activities. 

SUGGESTION 



School officials should consider determining 
teacher needs by making an annual plan of 
operation based on: 

0 a reexamination of goals 

0 a reexamination of means for accomplishing 
these goals 



Once the decision has been made to hire new teachers, it 
is then necessary for each principal to specify the 
number and types of teachers desired for his school. This 
may be done on a special requisition form which includes 
a brief job description to define special requirements of 
the position. The decision to approve a given position 
is usually made by central office personnel. 

NOTE 



A oleav speoifioation of desired teacher 
attributes assists those who do the recruiting 
and initial screening to locate and attract 
the type of teacher needed for a given position. 



Even with careful planning, there are times when over- 
hiring occurs, i.e., when more teachers are hiJC'ed than 
needed to accomplish the prespecified goals. When this 
happens , consideration should be given to the following 



When 

Overhiring 

Occurs 



Present 

Practices 
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possibilities for increasing the long-range effectiveness 
of the system: 

• establish special training sessions for people 
having difficulty or who need to acquire special 

S/V. JL ^ 

• reassign certain persons on the staff to accom- 
plish special needs, e.g., research, materials 
development, planning, new policy development 
in a special area. 

Although decision problems would be created in determin- 
ing who should be involved in the special activities, 
this procedure is potentially more beneficial than 
simply modifying the pupil teacher ratio. 



Determining Teacher Requirements for Specific Positions 

In the preceding discussion, it was noted that a requisi- 
tion form generally includes a brief description of 
special requirements for a position; this is beneficial 
both for recruitment and for initial screening of appli- 
cants. However, prior to making final decisions regarding 
teacher applicants, it is also beneficial to develop fully 
the teacher requirements for specific positions, which 
include a consideration of: 

• situational factors which affect requirements 

• development of a position analysis 

• who should be involved 

o problems of measurement 

Situational Factors . The major situational factors affect- 
ing teacher performance are: 

• pupil characteristics , including measures such as 
attitude, interests , abilities , motivation, morale , 
prior learning 

• principal characteristics , including his orientation 
to change, ability and interest in helping teachers, 
human relations and organizational skills , decision- 
making style 

• colleague characteristics , including general esprit , 
attitude toward newcomers and change, willingness to 
help new teachers and to plan programs cooperatively. 

5 
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Adaptability 

to 

Situations 



Purposes 

of 

Position 

Analysis 



The adaptability of teacher behavior to a specific 
situation is important for production of desired pupil 
outcomes. However, this adaptability is not exhibited 
in all teachers. Therefore, the number of different 
situations in which a teacher is effective depends on 
that teacher and his adaptability. It becomes necessary, 
then, to describe the general behaviors to be met by all 
teachers in a given school district, yet allow for indi- 
viduality oE the persons involved. The situation must be 
described fully enough that judgments can be made to 
determine whether a candidate is likely to meet the 
general requirements and adjust to the specific require- 
ments of the position. 

One situational factor which must be considered is the 
degree of change which is occurring in a situation and 
how much change is to be expected. It is true that 
jobs change men, but it is also true that men modify 
jobs; both of these factors imply there is need to 
examine how the last person in a position changed it and 
what effect the position had on the person. Such infor- 
mation is beneficial in making judgments regarding 
individual applicants, since ignoring unique situational 
factors tends to reduce effectiveness of predictions. 

Position analysis . The purpose of a specific position 
analysis is to gather and analyze data about the nature 
of the position so the personnel worker can: 

• identify what is important to teaching success — 
in that situation 

o ident ify how these important aspects can be 
measured 

• communicate to prospective new teachers the major 
and minor aspects of the position 

• infer what information might be collected to help 
predict the success of the applicant if selected 
for the job. 

The basic question to be answered is: ”What must an 

individual do, what must he accomplish, and what character- 
istics must he possess in order to be a successful teacher 
in this position?** The information presented in this 
section deals primarily with what is considered to be 
important in teaching and what information is likely to 
predict these important aspects . 
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What is 
Important? 



Include 
in the 
Position 
Analysis 



In teacher selection, the value of teacher performance 
is judged in terms of behavior of the teacher or result 
of his behavior. In either case, there is an attempt 
to predict indications of success. 

If one is concerned with the result of teacher behavior, 
he must decide what aspects of student gain should be 
used. Should one consider habits of work, modes of 
thinking, or merely the knowledge attained? Many people 
will be interested in all of these; but if this is not 
specified, selection procedures can lead to poor results. 

Some people have attempted to find ultimate and unchanging 
indicators of successful teaching which apply to all sit- 
uations. This search has been almost universally 
unsuccessful, due to the fact that: 

9 situations differ 

9 situations change (and therefore the required 
tasks change) 

9 the nature of teacnxng is very complex 

9 many different aspects of a job contribute to the 
success of a person on the job 

9 not all of these aspects of the job are related to 
each other 

9 different procedures can sometimes accomplish the 
same results. 

SUGGESTION 



Because of the complex and changing nature of 
each teaching position^ an accurate and detai-led 
position analysis (developed for each position) 
aids in making decisions in the selection of 
teachers. 



Such a position analysis should include: 

9 the general nature of the position , 

including the broad goals of the system, the organi- 
zational structure of the system, and the general 
expectation of the teacher in and out of the 
classroom 
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Information 
Used to 
Predict 



• th6 static and dynamic features of the positi on, 
including what is important at the beginning, what 
is likely to change and what will affect these 
changes, the nature of the students, and how the 
position will be affected by other people 

• the teacher behaviors required and desired , 
in the classroom and out of the classroom 

• the teacher characteristics sought , 

including aptitudes, skills , social requirements , 
interests, and physical requirements. 

In making judgments regarding which teacher behaviors 
should be specified for a given position, the following 
should be considered : 

• A limited number of teacher behaviors should be 
identified. 

9 Behaviors should be able to be observed accurately 
by individuals other than the teacher, should be 
specified precisely, and should differentiate 
among teachers. 

o The manner in which activities are organized., the 
choice of content, the types and sequences of 
activities, and strategies used in teaching are all 
means of producing effects on learners. 

As one studies the types of behaviors and the character- 
istics needed, there should be a conscious attempt to 
determine what variables will help to predict which 
candidates are likely to display these behaviors or char- 
acteristics. In doing this, the. following should be 
considered : 

o What information could be attained regarding the 
abilities of the individual? Are there indicators 
of his potentialities? What would help to predict 
later development? 

o What kinds of experiences can be documented and what 
will various experiences indicate? Does a certain 
pattern of experiences indicate a long-standing 
predisposition toward certain behaviors? 

o What information can be acquired regarding his 

human relations behaviors? How do others view him 
personally? Is he likely to get along with various 
types of people? 
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• Wl'iat information can be acquired regarding his 
breadth of knowledge ? How does such knowledge 
relate to the tasks expected of him? 

• What is known about his aspirations ? Is he 
interested only in a "job”? What is likely to 
motivate him on a long-range basis? 

Involvement of people . Since who is involved in making 
decisions (regarding expected outcomes in a given situa- 
tion and what behaviors are likely to precipitate those 
outcomes) is likely to determine not only how good the 
decisions are but also how satisfied people are with the 
decisions, it is very important to de,/elop a clear 
strategy for involvement of people . Many times , such 
considerations primarily involve a determination of 
whether the principal or the central office personnel 
will decide on specific expectations. When building 
principals make decisions regarding criteria, there is 
likely to be much more diversity in characteristics being 
sought than when the central office controls the final 
determination . 

Ideas to However, the strategy for involvement of people should be 

Consider much broader than simply indicating who makes the final 

decision, and the following points should be considered 
when determining what items will be included in determin- 
ing teacher success and how much weight will be assigned 
to each item: 

• P^^gnts can indicate what expectations they have for 
their own children 

0 students can indicate what motivates them and helps 
them to learn 

• teacher c can indicate behaviors desii.'^ble for a 
given position — and this can be done in collaboration 
with 

• principals , who can indicate expectations for general 
teaching behaviors, out-of-classroom behaviors, 
general results of bcnavlor 

• specialists (e . g . , consultants, supervisors , depart- 
ment heads) can ind cate special skills needed for 
given content areas 

• central office administrators can indicate general 
organizational expectations and long-term needs . 
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Advantages Involving many people in determining what is desirable 
iu teaching positions is likely to help determine 
teacher behaviors and student outcomes in terms that are 
relevant to the specific situation , i.e., the involve- 
ment of many people develops acceptable or socially valid 
standards for teaching success. The criteria for success 
should be acceptable to a wide variety of people who are 
affected by teacher behaviors, including students, par- 
ents, teachers, administrators and other members of the 
community . 

However, when acceptable validity is determined, it should 
not be expected that this description of success will be 
maintained over a long period of time. There are indica- 
tions that these specifications change rather rapidly 
due to changing student and parent populations , societal 
changes, and changes in conditions of work due to new 
technology and prof es sional developments. 

CONCLUSION 



Specif ioations of desired student outcomes 
and teacher behavtors should be reviewed 
frequently with a variety of people affected 
by teacher behaviors* 



Observing 

Teacher 

Behavior 



Problems of measurement . The problems of measuring 
student o'»tput are discussed much more thoroughly in 
texts concerned with tests and measurements in education 
and psychology, and no attempt will be made to cover this 
topic here. However, certain problems encountered in 
measuring student growth are worth considering at this 
point . 

When measures of student growth are what is desired, 
what is the justification for observing teacher behavior, 
student-teacher interaction, or of using student percep- 
tion of teacher behavior? Perceived teacher behavior 
intervenes just prior to pupil change; therefore, it is 
something which might justifiably be expected to affect 
that change. Based on this view, much research has been 
done in the past ten years on developing observation 
schemes which will accurately measure certain aspects of 
teacher behavior and of classroom interaction. In 
developing such procedures, there is an emphasis on: 

• obj ectivity , which can be determined by using 
different observers at the same time 
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• consistency , which can be checked by asking the 
same observer to observe the same phenomenon at 
different times (this requires recording the 
phenomenon on audio or TV tape, or filming it) 

• accuracy , which involves using different observers 
at different times (with the same observable sets). 

In any case, it is necessary to develop not only an 
abixl**y to measure, but a willingness to measure teacher 
behavior as well. Unless those responsible for measuring 
teacher behavior are actually willing to spend the time 
necessary to measure accurately and reliably, the develop- 
ment of procedures is futile. 

Criticisms Although observation techniques have been developed con- 

siderably during the past years, they are still criti- 
cized by many because: 

• There is no general consensus of what good teaching 
is 

• The situation is artificial with an outside 
observer 

• Adequate observation is expensive. 

However, each of these criticisms has been adequately met 
with count erargument s , and observation o f classroom 
procedures is the best route to adequate diagnosis of 
why certain procedures yield specific results in local 
situations. Without adequate information regarding pro- 
cess in the classroom, teachers may make changes when 
they are not warranted or not make changes when they are 
needed . 

What to What should be observed in a classroom? It is generally 

Observe agreed that the following should be observed: 

• teacher characteristics 

• content being taught 

• teaching strategies whicli are: (a) specific and 

unique to the content, and (b) general, i. e. , not 
specific to the content being taught. 
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^ NOTE 



What is 
Recruitment? 



Effective 

Recruitment 
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The content being taught and the teachii'ig 
strategies being used may be determined by 
the behavior of the teacher; however ^ the 
teacher characteristics must be inferred 
from the behavior exhibited^ 



The last note is cc*ncerned with the distinction between 
patt erns of instruction being used by a teacher and the 
teacher as a person. Notice that both depend on observa- 
tion of the behavior of the teacher, but one must be 
inferred while the other is observed directly. 

In considering observation procedures, it should be 
emphasized that intermediate criteria (such as teacher 
behavior, which intervenes just prior to pupil learning) 
are significant only as they are relevant to the ultimate 
goal (generally some change in behavior, or eventual 
behavior, of students). What is ultimately desirable 
must always be determined on rational grounds, i.e., via 
judgment of some individual or group. 

Recruitment of Teachers 

Recruitment of teachers includes all activities that are 
used to encourage teachers to apply for positions. Many 
people misuse the terms "recruitment" and "selection" by 
considering them as synonyms; for example, when they say, 
"There is no teacher selection problem now— -there are 
plenty of teachers available," what they really mean is 
that the aspect of recruitment (which is only one part of 
selection) which requires that an adequate q uantity of 
applicants be available does not appear to be a problem. 
Sometimes these people even overlook other problems of 
recruiting, e.g., adequate quality of teachers and number 
of teachers will special skills or background. 

When a surplus of teachers exists, school S3^stems empha- 
size rigorous selection procedures. However, when there 
is a shortage of candidates, school systems must develop 
and maintain an effective recruitment program as a 
prerequisite to good selection . 

In order to attract an adequate number of well qualified 
teachers, it is necessary to establish good public rela- 
tions procedures throughout the selection process. Since 




Guidelines 



Cost 

Analysis 



applicants form definite impressions and may become 
’’recruiters*’ for the school system even though they 
are not hired, there should always be an effort made 
to show potential applicants that the district has some- 
thing to offer now, in the near future, and in long-term 
career goals. They should be made aware of the profes- 
sional possibilities for growth and successful fulfill- 
ment of goals. In addition, throughout the selection 
process, one should : 

• show consideration for all applicants and avoid 
any practice that i .ight create ill will 

• not create false hopes 

• develop ways of saying ”no” that are sincere and 
yet as unobjectionable as possible 

• show interest in professional development of 
applicants and in their being placed in the 
position which is most advantageous to them. 

Specific guidelines which districts have found to be 
helpful in recruiting teachers include; 

0 Devise ways to help placement directors get 
acquainted with the school system 

0 Provide personnel and curriculum materials to 
college and university classes as a means of 
getting students acquainted with the school 
system 

0 Develop ways to contact and communicate with 
experienced t eachers 

• Use multiple media teaching procedures to 
acquaint applicants with details of the system 

• Explore unusual advertising means to attract 
specialized personnel 

Some recruitment procedures cost more than many school 
districts want to spend on this aspect of selection; 
however , industry generally spends considerably more 
than schools on recruitment and considers it money well 
spent because of reduction of turnover and higher 
productivity. The cost of hiring the wrong candidates 
can be higher in terms of supplementary training, wasted 
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salary, adverse public relations, and lost productivity 
than the cost of more extensive recruitment. 

SUGGESTION 



School districts should analyze the results of 
their recruitment programs as well as the costs. 
Increased initial costs may produce a more 
effective recruitment program which establishes 
better long-range results and reduces costs 
through smaller turnover . 



Collection of Information 



Sources of The primary sources of information for the selection of 
Information teachers include: 

• written documents , including letters of applica- 
tion and inquiry , the resume^, application forms , 
recommendations, placement office materials, and 
certificates 

o interviews 



• tests , including paper and pencil, and situatT <^nal — 
such as those which require performance assessment 
in simulated situations . 



Although many of these sources have proved to be bene- 
ficial to specific school districts and businesses, 
local school districts should conduct research to justify 
the use of any of these sources of information for 
predictive purposes in the local situation . 

Sequence of The general sequence used in collection of information 

Information regarding candidates varies considerably from district 

Collection to district and from business to business. This sequence 
is affected by policy regarding the degree of central 
office control of decision making. The functions of the 
central office and the school building personnel are 
different ; for example , the central office personnel are 
responsible for coordination of the procedures used 
throughout the organization. If uniformity of procedures 
is desired, they must initiate procedures to produce this 
uniformity (however, this does not mean that they must 
collect all information). Building personnel, such as 
principals, are in a better position to relate to the 



14 



27 



personal concerns of the applicants; consequently, they 
are involved at different times in order to perform 
functions related to their peculiar contributions. 

If candidates are screened out of consideration before 
all information has been collected on them, i.e,, if 
a "successive hurdles" strategy is used, selection 
instruments which provide rough screening of applicants 
should be used first and those which provide finer 
discrimination among candidates should be used In later 
stages of selection. Also, cost should be considered in 
determining sequence of data collection , For example, 
if an application form can be used inexpensively as a 
rough screen, the applicant pool can be reduced before 
using the more expensive technique of interviewing for a 
finer discrimination among applicants , 

NOTE 



There is a need for more definitive information 
on the costs of different data collection 
devices and their contributions to discrimina- 
tion among applicants. 



A Caution When an item of information is used to eliminate an 

applicant, one should be certain that the predicted 
behaviors are actually necessary to the teacher’s satis- 
factory performance; further, the behaviors should be 
necessary for initial work rather than behaviors which 
can be attained after the position is taken. For 
example, assume that extremely good organization is 
considered necessary for success in a particular position, 
and evidence from practice teaching indicates that a 
given applicant is not very well organized. An error 
might be made in immediately eliminating such an appli- 
cant, since he might respond favorably to assistance 
once he has been hired. This might be especially true 
if his attitude and interest are good, but he has had 
little prior assistance or training. 

Since research indicates that the format in which informa- 
tion is presented can improve decisions, school district 
personnel should examine ways to improve presentation of 
information to the person making decisions, 

• Small schools may use a combination of a visible 
card record file and a summary document which 
have been compiled by hand. 



Format of 
Information 
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• Larger districts may use a computer to prepare 
summary documents and status reports on 
applicants. 

• Medium- size districts may share computer 
services in cooperation with other districts. 

As information is being examined (regardless of the 
source), personnel men in business and industry, 
governmental agencies , and school districts consider the 
following as negative information and the person who 
exhibits them a poor risk: 



• negative attitudes toward supervisor and 
organization 

• excessive illness or tardiness 

• poor strength or energy 

• poor motivation as indicated by a need for a lot 
of follow-up or a need to be pushed to do a job 

• slow learning as indicated by a lot of mistakes, 
trouble understanding instructions, impulsive 
actions, or unwillingness to think independently. 



Written documents . Letters of application (some of which 
includ^ a resum^) are often received on an unsolicited 
basis. A well-formulated procedure for handling such 
letters is needed ; the following are suggestions developed 
from practices in schools and businesses: 

• Acknowledge all letters , provide specific informa- 
tion 

• If no vacancy is available, inform the applicant 

• If you keep names on file even though there is no 
vacancy, acquire the completed application blank 
in order to make an initial screen 

• If the applicant comes in personally, let him complete 
an application blank, interview him briefly and let 
him know what procedures will be followed. 

Businesses and industries use the application form (and 
the personal history record) to predict job success much 
more than schools; in many cases it has proved to be a 
beneficial aid. 
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SUGGESTION 



References 
and Recommen- 
dations 



School distr^icts should capitalize on the 
potentially beneficial predictive information 
contained in the application form. 



In school personnel selection, application forms tend to 
be lengthy and contain much information that is not used 
for prediction purposes. Sometimes this length is due 
to the fact that a general purpose form is used for 
all applicants for all positions. Industrial organiza- 
tions tend to devise forms for specific positions in 
order to reduce the time spent by applicants in completing 
the form and to make it more usable for prediction. The 
implication of such procedures is that local districts 
should investigate the possibility of devising different 
forms for different types of positions, e,g. , secondary 
and elementary teachers.^ or team teachers and self- 
contained, etc , 

Application forms and documents from placement offices 
generally have a place where references may be listed; 
these references are usually asked for recommendations. 
Likewise, immediate past employers are usually asked for 
recommendations. Two methods are used for checking 
references : 

• written (most businesses and many school districts 
have a checklist of items for which information is 
to be provided) 

• telephone or visit (many businessmen use a form to 
record the information acquired). The benefits of 
the telephone check are that it saves time, reduces 
ambiguity because of the two-way conversation, 
amplifies incomplete or unfavorable information , 
and is often the easiest way to get to a supervisor 
(especially where mail is shunted to a central 
personnel office) , 
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NOTE 



Problems 



Suggestions 



Industry checks references before an applicant 
is employed^ generally obtaining information 
from the immediate supervisor rather than the 
personnel office ayvd. checking on: 

0 stroyig and weak points of applicant 

0 how he performed, on prior jobs 

0 why he left prior position(s) 

0 work habits. 



Although recommendations by persons who know the 
applicant well can be a good source of information, 
there are problems which can reduce the benefit of 
recommendations for predicting performance . Among 
the problems are : 

n non-discriminating descriptions 

o validity varies according to source (especially 
in terms of how well the individual is known) 
and according to the quality of the description 

• descriptions often include mainly conclusions 
rather than information regarding performance 
in specific situations. 

Since these problems with recommendations exist , most 
businesses and schools have taken steps to remedy some 
of the problems. Among the most useful suggestions are 
the following : 

• Inquire only about those qualifications that c^re 
most important . 

• Seek information from a variety of sources, 

• Clarify who is seeking the information and how 
the information is to be used . 

0 Provide information to the reference regarding 
the position to be filled • 
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Intervi ews 



Variation 
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9 Seek specific information regarding behavior and 
results of behavior as well as the exact nature 
of the prior position . 

• Seek information regarding the conditions under 
which the respondent observed the applicant . 



Interviews ^ Interviews have two purposes: to acquire 

information from applicants and to provide information ^ 
them; the type of information being sought and provided 
determines the structure and cont ent of the interview. 

For example , if the supply of teachers is scarce , much 
more time might be spent on "selling’’ the applicant on 
the job than when the supply of teachers is greater. 

Also, some time might be spent in each interview giving 
the applicant information about the position and working 
conditions, since it is beneficial to create goodwill 
toward the school district even if the applicant is not 
selected. 

Because of differences of emphases on collecting informa- 
tion and providing infurmation to applicants , considerable 
variat ion exists in interviewing practices throughout the 
country. These differences exist in: 

• structure , ranging from very flexible to highly 
structured interviews with questions determined 
prior to the interview 

• number of interviews which each applicant must 
have prior to hiring and the sequence of these 
interviews 

• use of group interviews (primarily used by 
industry) 

• variety of people who are involved in interviews, 
ranging from students to citizens’ groups, with 

a general increase in the use of teachers and non- 
professionals 

• amount of training given to interviewers 

• relation of written information to interview (some- 
times ignored prior to interview, sometimes studied 
carefully. ) 

• how interview information is analyzed (sometimes done 
immediately by interviewer, sometimes later by 
another person). 
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Determine Research does not provide definitive information 

Objectives regarding all aspects of interviewing; however, it does 

and Form indicate that when the objectives of the interview are 

vague and ill-defined and when no form is given to the 
interview, there is little reliability or validity to 
the conclusions reached , 



SUGGESTION 



When developing interview procedures^ care 
should be taken to define objectives care- 
fully and to specify the type of structure 
which will be used^ 



Stress The stress interview provides a particular type of 

Interviews structure and attempts to simulate tensions similar to 

those on the job in order to observe the applicant’s 
tolerance for strain. Research does not show that the 
stress interview is any more successful than any other 
type. Since the stress could have an adverse effect on 
the applicant and therefore affect recruitment, it is 
suggested that any district which desires to try the 
stress interview develop careful plans for implementing 
the procedures. Further, research should be conducted to 
determine the effect of the interviews on applicants who 
are hired and rejected as well as the effect on selection 
success. 

Most interviewers combine directive questioning and non- 
directive procedures which use broader, open-ended 
questions. Following are general guidelines used by man}^ 
organizations : 

• Put the applicant at ease by asking seme specific 
questions and getting him to talk, 

• Never criticize or register disapproval; do not 
give applicant indications of what you consider to 
be a good response to a question • 

m Ask some questions which require the applicant to 
structure his own response and resist talking 
during pauses to give him time to reflect. 

• Keep the interview focused by restating questions 
and be alert to areas that need additional probing. 

• Allow sufficient time for each applicant to acquire 
adequate information about the position v 



Guidelines 

for 

Interviews 
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• Provide information in written form the applicant 
to read prior to the interviev' to save inVerview 
time for questions. 



Errors in Research shows that several types of errors can be made 

Questioning in interviewing. One source of errors is the type of 

questions used and the manner of questioning; some of the 
more common errors of questioning include: 

• asking questions which can be answered "y^s" or ’’no” 

• unimaginative questions 

• leading questions 

• qu‘="-stions which reveal interviewer’s attitudes 

» questions x^^hich have been answered in written 
document s 



Other 

Errors 



Research 

Needs 



• questions which do not relate to the task at 
hand . 

In addition to errors of questioning, most other errors 
which adversely affect the interviewing process are 
related to the judgment of the interviewer. For example, 
research has shown that relatively minor items may 
influence the interviewer unduly. Such items as a super- 
ficial resemblance of the applicant to someone the inter- 
viewer has known in the past, a reaction to nervousness 
on the part of the applicant, a fear of weakness in the 
applicant, a projection of interviewer characteristics 
into the applicant, or empathizing with the applicant may 
bias the interviewer. When this occurs, it implies that 
the interviewer needs training and practice, and informa- 
tion should be fed back into the system so that corrections 
may be made. 

In addition, there is a need for research information 
regarding the type of people who can make good judgments, 
when these people tend to reach closure on these judgments, 
and how others can be taught to make r^etter judgments. 

Many people believe that they can make good judgments 
about other individuals in a very short time; but research 
has not substantiated this for most interviewers. 
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SUGGESTION 



Selection 
of Inter- 
viewers 



■^n addition to selecting interviewers carefully, atten- 
\ Ion should be paid to providing them with training and 
practice. Smaller school districts frequently involve 
principals in interviewing, 

SUGGESTION 



Regardless of the size of districts ^ when 
principals are involved in interviewing j 
school districts should consider following 
the general practice of industry (and of 
some school districts) of providing relatively 
extensive training in interviewing procedures. 



In summarizing informatior, regarding interviewing, it can 
be said that no generalizations can be made as to the 
"goodness" of interviews because the interviewer is part 
of the measur-'ng instrument as he judges each candidate. 
In fact, the validity of interviews is frequently challen 
ged by research results which indicate that not all inter 
viewers are good. It appears that the validity of the 
interview depends on the skill of the interviewer, the 
situation, and the nature of the applicants. Training of 
interviewers has been effective in preventing errors and 
increasing discrimination of interviewers, but additional 
research on the effect of the interview, separate from 



Summary 

Regarding 

Interviews 



Training 

Interviewers 



Not all people are equally adept at interviewing . 
When selecting interviewers, the following quali- 
fications should be considered: 

0 alertness to cues 

0 ability to make fine distinctions, per- 
ceive accurately 

0 ability to make immediate and accurate 
records 



0 willingness to use criteria established 
by the orgayiization 

0 ability to suppress biases 
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that of the interviewer, should help to improve the 
effectiveness of the procedure, 

SUGGESTION 



developing local interviewing procedures^ 
there is a need tj: 

m develop ways to test interviewing skill 

0 select those who seem to do intervie\)i 7 ig 
well 

0 develop means to train interviewers by 
helping them to acquire information (a 
measuring act)^ process information (by 
analysis anc interpretation) ^ and eval- 
uate information (in terms of pre- 
determined standards) . 



How Good * 
Are Tests? 



Factors to 
Consider 



Tests , Paper and pencil tests are used much more by 
businesses and industry than by schools, but industry 
does not use as many tests with professional level 
people as with other employees. It is generally agreed 
that paper and pencil tests which are used for selection 
purposes are neither good nor bad, but that they should 
be selected in terms of what they will contribute to a 
decision in a local situation. This means that one 
should determine whether the test has a proved relation- 
ship to specific job success, since otherwise hiring 
high scorers may simply result in hiring those who are 
good test performers . 

Factors to consider in deciding whether to use paper and 
pencil tests for selection purposes include : 

• potential contribution to the selection decision 

• impact on potential applicants 

• impact on employee morale 

• cost 
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SUGGESTION 



One should always weigh the potential improvement 
of the prediction of success or failure against 
the increased co^t of testing. 



Assessment A reiativel/ recent development being used by business 
Centers and industry to appraise a candidate’s potential for 

success is the assessment center approacht This procedure 
combines simulation exercises, observation of behavior in 
small groups, writing, problem rolving, and paper and 
pencil tests. 

For example, the popular ”in-basket" test, which presents 
the candidate with a typical day’s problems, not only 
probes his technical knowledge and his familiarity ^-ith 
company policy but also his ability to plan his work, to 
deal with emergencies in the absence of his superior, and 
to delegate or postpone solution of less important prob- 
lems, In industry, assessment centers are often used to 
test employees for ’’management potential.” The procedure 
has met with some success in industry and has shown enough 
promise to merit investigation for possible application 
in education, especially by large school districts and 
colleges of education. 



Institu- 

tional 

Decisions 



Decision Strategies 

The first idea which must be grasped by a person making 
teacher selections is that he is involved in making insti- 
tutional decisions; difficulties arise when this fact is 
overlooked. Institutional decisions employ a common 
value system for a set of decisions and an attempt is 
made to use the same criteria over a sustained period of 
time. 



CAUTION 



When decision makers allow the value system 
of the applicant to sway their judgment 
regarding organizational needs, their decisions 
are likely to he inconsistent and not as bene- 
ficial to the organization. 
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Sequential 

Hurdles 

Strategy 



Involving 

People 



For example, if a given applicant does not fit tlie job 
particularly well but has a great need for a job and is 
willing to take risks with regard to his own success, 
there may be a tendency to hire the individua] out of 
sympathy for his need. However, unless the extra motiva-* 
tion (which might be indicated by the need of the appli- 
cant) compensates for other shortcomings which are 
evident, the decision to hire this person might not be 
in the best interest of *:he organization. 

Most strategies for selecting teachers allow decisions to 
be made at several points during the selection process, 
including any of the following ; 

• preliminary interview 

o application form 

• tests 

• interview(s) 

• check of references 

• written records of background 

• discussion with principal 

• physical examination 

• certification 

Although decisions may be made at any of these points, 
usually the decision is either not to hire the applicant 
or to continue seeking additional information. In effect, 
the strategy is a "sequential hurdles" approach, with the 
decision to hire the individual being made only after all 
information has been collected , 

It seems relatively obvious that a number of people might 
be involved in making decisions regarding applicants at 
the various steps indicated above . The init ial screening 
may be done by one person or one group, while the final 
judgment might be made by another. Likewise, the decision 
of fitness for hiring may be made by one individual, while 
the decision about placement may be made by another. 

Selection of teachers is a major personnel function; how 
it is done affects relationships with staff, students, 
and parents. The function of selection of teachers is 
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too important to not be systematically planned and well 
executed; therefore, one person in each school district 
should have the responsibility for organizing, coordina- 
ting, and directing the selection process. In larger 
districts, this is likely to be a full time position for 
an assistant superintendent or personnel director, where- 
as in smaller districts one person might have this as one 
of his several assignments, 

• The important point is that poor selection 
procedures should not be allowed to develop 
because of a lack of specification of who 
should be responsible for the function. 

Since the selection process is not a single-phase opera- 
tion (it consists of much more than simply making a choice) , 
one of the major responsibilities of the person in charge 
of selection is to determine who should be involved in 
what phases of the selection process and what should be 
the sequence of activities. 

Placement When a person is hired for a specific position, no 

separate strategies for placement are necessary; but when 
teachers are hired into a ”pool," special placement 
strategies are necessary. Such strategies in use in 
public schools include : 

• central office makes the placement decision 

• central office makes the placement decision on 
approval or recommendation of the principal 

• a committee makes the placement decision 

• the applicant has the opportunity to make a 
choice and be approved by the principal. 

Job Offer The decision of who is the most suitable of a number of 

applicants is a different decision from the one to make a 
job offer. The first decision involves predicting how 
each of a group of applicants will perform in a given 
position; the second involves predictions regarding whether 
applicants will apply and whether certain other applicants 
will accept the job if it is offered. The following points 
should be considered in making job offer decisions: 

o Are better applicants likely to apply within 
the time period when an offer should be made? 
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Errors by 
People 



1 rocedural 
Errors 



• If other applicants do apply, what is the likeli^ 
hood that this position will be attractive to 
them? 

• Should multiple offers be made? 

• Who will decide when to make an offer? 

In making decisions about the selection of teachers, a 
number of errors may be made related to the interpreta- 
tion of data. Some of these errors a7;e due to the people 
involved, while others can be traced to faulty organiza- 
tional procedures . Those which are most likely to be made 
by people include: 

o errors of rating such as leniency, halo effect, 
contrast , and central tendency 

• errors of logic 

• errors of rationality, including faulty rating, 
confounding, overgeneralization, and premature 
closure. 

Some of these errors may be reduced by training, but much 
research needs to be done to determine how to reduce human 
error in decision making . Other errors by people may be 
reduced by selecting people who make fewer of these types 
of errors. 

Poor organizational procedures can contribute to errors in; 

• h uman relations , such as careless treatment of 
candidates 

• decision making strategies which overly depend on 
expert opinion, which misjudge job requirements, 
or which overlook the need to select people com- 
patible with personnel within the organization 

• policy , such as that which makes stop-gap appoint- 
ments 

• communication , because of inade'"acu-e means to process 
applications and documents , 
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CONCLUSION 



On the basis of the research conducted to dcte^ 
the following conclusions appear to be justified: 

0 Training and experience of the decision 
maker influences hit interpretation of 
information. 

0 Ability to interpret data is not related 
to sex. 

0 More accurate judgments are made by those 
who do not become emotionally involved or 
who are socially detached. 

0 Use of multiple raters tends to improve 
predictions . 

0 The selection decision is improved by 
using a single page summary document 
and by providing instructions on how to 
process information. 



Controls of the Selection Process 



To control the teacher selection process, it is necessary 
to analyze sources of error and initiate procedures to 
eliminate errors. In most cases, errors result from 
improper goals, inadequate procedures, and/or improper 
implementation of procedures. Since errors may occur 
at any stage in the selection process, a periodic exami- 
nation should be made to determine if the process is 
functioning well at each of the following stages; 

• determination of teacher need 

• determination of teacher attributes desired 
• recruitment 

• collection and processing of data 
• making predictions 

• developing strategies for choices 
• final selection 
• placement. 
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In addition to clieckln^ errors, one sliould nlso examine 
outstanding successes in an attempt to determine what 
procedure was used, liow tlie predictions were made, and 
who was involved. Successful selections may yield as 
many clues to proper functioning of the selection process 
as errors. 

To investigate possible alterations in tlie implementation 
of tlie total selection process, one sliould examine the 
collection and processing of data hy looking at: 

• ease of administration of information collection 

• ease of scoring of tests or of rating of observa- 
tions in interviews or written documents 

• ease of interpretation and application 

• reasonable cost 

o proper construction of records , instruments . 

Measurement specialists caution that there is no such 
thing as general validity for any measure, since its 
usefulness has to be proven for everv situation in wliich 
it is to be used. Research indicates that correlations 
among two or more variables tend to change from time to 
time because of the fact that both vjorking conditions 
and personnel in schools change . 

^ corrcLUsioN 



The imptioation of research findings and 
advice from measurement specialists is that 
any measure of success of a selection process 
is likely to he only temporary. This means 
vhat the value of the procedure should be 
checked periodically . 



Finally, it should be emphasized that the ultimate value 
of a district’s selection process depends on how well it 
predicts the success of teachers in meeting a given school 
district’s needs. For this reason; 

• Objective investigations must be conducted at the 
local level to determine the effectiveness of the 
selection process* 
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• Local districts cannot depend entirely on research 
conducted outside the district to evaluate their 
selection procedures (since general research gives 
clues to what might be tried, but can not tell what 
is actually effective in a given situation). 
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ABSTRACT: 



EVALUATION OF TEACHERS 



The reasons for establishing comprehensive and systematic 
teacher evaluation procedures are to improve learning con- 
ditions for children and to facilitate administrative deci- 
sions. Evaluation procedures are most effective hihen they 
are cooperatively vlanned by teachers and administrators 
h)lio receive Ui.-...'' stance from specialists , consultants, 
parents, and students. All personnel involved should 
understmd ihe purposes of evaluation, the nature of the 
procedures, and the roles of the various people involved. 

The purposes of teacher evaluation include: 

0 to improve instruction 
0 to reward superior performance 
0 to supply information for modifying assigriments 
0 to protect both the individual and the school system 
0 to validate the selection process 

0 to provide a basis for career planning of ‘-■he teacher 
0 to facilitate self-evaluation. 

Evaluators should be well trained to assist in the follow- 
ing stages of the evaluation process: 

0 goal setting conferences 
0 observation and information collection 
0 post-observation conferences , communication 
0 decision making 

0 assessment of the evaluation process. 

Problems are encountered in teacher evaluation when there is 
an emphasis on fault- finduig rather than helping teachers; 
when prejudice, bias, and poor judgment are used, in collecting 
and analyzing information and when communication is not open 
and honest. The most effective evaluation plan is one vh-'-ch 
contributes to higher teacher morale and improves learning 
conditions for boys and girls. Questions discussed in the 
followring pages include: 

0 What are the implications of multiple purposes for 
evaluating teachers? 

0 Who should be involved in planning for evaluation? 

0 Who should determine teacher behaviors desired 
and learyier outcomes desired? 

0 Who should be involved in evaluating teachers and 
in what way? 

0 How should evaluators be trained? 

0 What problems should be avoided? 

0 What decisi-ons should be made on the basis of 
evaluation? 

0 How can the evaluation process be analyzed for 
improvement? 
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Goa 1- Set: ting Conf erences 



The 

Question 



It is assumed that the teacher evaluation process consists 
of the following stages: 



• goal-setting conferences 
r observation and information collection 



• pos t-observation conf erences , communication 

• decision making 
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• how the total evaluation program should be p'*anned 

• what purposes for evaluation are 

• what problems are likely in teacher evaluation 

• how evidence for determining when deisrable out- 
comes occur can be established. 



l^at is Evaluation? 



All teachers are evaluated. Re 
system for evaluation is, what 
analyzed, how often formal repo 
are evaluated and they are eval 
are evaluated by students , pare 
istrators and supervisors, and 
is not whether teachers should 
cannot be avoided ; rather the q 
the evaluation should be in ord 



gardless of how^ formal the 
evidence is collected or 
rts are written, teachers 
uated rather often. They 
nts, other teachers, admin 
the public . The question 
be evaluated, since this 
ues tion is hoxv" systematic 
er to be most effective. 



This is the question facing administrators and teachers : 
how can an evaluation system be designed in order to be 
effective? Effectiveness must be in terms of certain 
purposes desired for the school district; and the design 
of the evaluation system should include some way to collect 
and process information, a way to communicate with peon.''e 
concerned, and ways to make decisions and assess how well 
the eva] ration system works. 
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Precision 



An Example 



Yet, evaluation is a nebulous problem. It is difficult 
to grasp in its entirety, a little different from school 
to schcol cxnd from district to district. Because of this, 
•lome people approach the problem with the attitude 
Charlie Brown expressed in a Peanuts cartoon • The dialogue 
was as f ollows : 

Lucy: How are you doing in school these days, 

Charlie Brown? 

Charlie Brown: Oh, fairly well, I guess... I’m 

having most of my trouble in arithmetic. 

Lucy: I should think you’d like arithmetic. 

It’s a very precise subject, 

Charlie Brovui: That’s just the trouble. I’n) 

at my best in something where uhe answers 
are mostly a matter of opinion. 

Many people consider the evaluation of teachers to be just 
that: mostly a matter of opinion. The problem underlying 

this viewpoint is tne question of precision — how precise 
a principal desires evaluation to he, how precise the 
superintendent and school board desire evaluation to be, 
and how precise teachers desire evaluation to be. VJhat 
are some of the implications of varying degrees of pre- 
cision? 

For examiple, suppose you wanted to measure the length or 
the circumference of a small object. Three different tools 
might be used: a piece of string, a ruler, or a micrometer. 

The string has obvious advantages and disadvantages: it 

is cheap , plentiful, easily available, expendable , and 
easy to use or explain to an untrained person. On the 
other hand, it tends to wear out and stretch with age and 
use, it lacks accuracy because of i^.s relative elasticity, 
and different people tend to retch it to dif f ering 
lengths when they use it for measuring. 

The ruler has definite advantages over the string: it does 

not stretch (increasing what is called the reliability of 
tlie meacurement) and it tends to last longer than the 
string. However, rules c^re not so available as string, 
and untrained people make mistakes with them. 

The micrometer is much more expensive than either of the 
other tools, is less accessible, and is difficult for 
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Definition 



Evalu'ite 
Program or 
Individuals? 



some people to use. Yet, it has an accuracy that is 
necessary for some jobs — an accuracy that cannot be 
obtained with the otlier tools — and in the hands of a 
'rained operator it is seldom misused. 

Tlie analogy to measurement in the evaluation process 
appears clear. Some measurement devices have character- 
istics similar to the string; others , while more accurate, 
have some of the disadvantages of the ruler; while still 
others have the specialized uses of the micrometer. 
Therefore, measurement devices and procedures are chosen 
according to liow precise one desired to be, and what types 
of resources are available; consi lerable precision can be 
obtained if it is desired. 

Note the distinction between measurement and evaluation. 
"Measurement" is many times used synonymously with 
"assessment" and is related to the quantified or quasi- 
quantified description of events, behavior, or outcomes. 
However, evaluation (in the context of evaluating teachers) 
has to do with j udgments relating to the "goodness" of 
teacher behavior and/or results of that behavior in light 
of agreed-upon objectives. These objectives usually are 
approved (either tacitly or explicitly) by a particular 
school system and community, giving the objectives a type 
of social validity for that community. 

There is a difference between assessing or evaluating the 
program of a school oi a school system and evaluating the 
individuals \jho contribute to that system. The former 
attempts to look at the output of the total system; the 
procedures used to accomplish those outcomes, and the way 
those procedures were implemented. The latter examines 
the contribution to the system, in terms of individual 
output, procedures, and implementation. This report is 
concerned with the evaluation of teachers rather than 
evaluation of systems of teaching vithin school systems. 

Tlie advantage to examining the effectiveness of teachers 
is that one can make better inferences about system 
performance from combinations of subsystem performance than 
he can make about subsystem performance from the total 
system performance. For example, if one knows something 
about each individual teacher^s performance in a building, 
he can infer something about the total performance of the 
school. However, knowledge of the total productivity of 
the school would provide little basis for inferring how a 
given teacher behaves. 



What is 
Involved 



Accurate measurement can occur without evaluation . How- 
ever, it is impossible for evaluation to be good without 
adequate measurement. In addition, the data collected 
via the measurement process must be adequately analyzed 
and interpreted if good evaluation is to occur. There- 
fore, good evaluation is preceded by ; 

• a determination of what is important (criteria) 

• measurement 

• analysis 

• interpretation- 

judgments made prior to these activities are likely to be 
unsound, (Note that it is assumed that the making of 
judgments regarding teachers is inevitable, ) The real 
issues are concerned with whether the criteria are 
appropriate and the data are sound , 

CAUTION 



In evaluating teachers^ the emphasis is on 
making judgments in relation to ohjeotiveSj 
not on judging the personal worth of people, 
(viewpoint: research staff) 



Formal evaluations should be analytic rather than compar- 
ative, establishing whether the teacher reaches various 
standards but avoiding attempts to compare the teacher 
with other teachers. The emphasis should be on helping 
individuals improve their contributions to the learning 
of school children rather than on taking punitive or 
controlling actions, making odious comparisons , or using 
questionable motivation techniques. 



Planning for Teacher Evaluation 

Suppose you had the responsibility for planning a teacher 
evaluation program for a school or a school district, 
Wliere would you begin? Whom would you invclve? What 
decisions would need to be made? How rapidly would you 
tr> to implement the procedure? 
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‘.)|) j oc L i ves 



People 
I nvol ved 



One of tlie first step 
evaluation program is 
impor tan t in teaching 
evaluation program is 
ti on of vliat is impor 



s to be taken in planning a tench 
to determine what is considered 
— in tlie situation for which tlie 
being considered. This determin 
tant in teaching allows one to : 



er 



a- 



o Gstablisli what is expected of teachers (as far 

as behavior and outcomes of behavior are concerned) 

• determine ways to measure what is happening 

• design ways to compare what occurs with what is 
desired . 



However, who will be involved in deciding what is impor- 
tant in teacliing? Certainly, one should involve teachers 
in this decision and in tlie total design of evaluation 
procedures , * The involvement of teachers in planning 
evaluation procedures is based on the premises that: 

• a better plan will develop 

0 there will be more teacher commitment to the 
procedures 



• teachers will know what they are to do 

• teachers \;ill know what will be evaluated. 

Tlieee premises are more than assumption or mere hopes. 

Sc’nool districts which have involved teachers in planning 
have found that output and procedural goals for individual 
teachers are better understood and attained when they are 
cooperatively developed and written in precise terms than 
when they are unilaterally dete: ined or written in very 

general language. Also, there is more commitment t> goals 
and procedures when goals are specific and attainable 
and when people who are to accomplish the goals are involved 
in establishing them. 



In addition to teachers, administrators should be involved 
in planning the teacher evaluation procedures. In fact, 
some propose that the design of an evaluation procedure 
should begin with an evaluation of how well principals and 
supervisors evaluate teachers. The insights gained frota 
this experience would form a firm basis for cooperation in 
developing a plan for teacher evaluation. 



*A more detciled discussion of involvement of other people 
begins on page 48, 
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Activities 



Time 

Sequence 



The reason for beginning with the administrative- 
supervisory staff is that the procedures for evaluation 
of professional personnel are the same, regardless of 
the level or position of the personnel. This does not 
imply that criteria will be the same, but that the methods 
of establishing criteria, data collection and analysis, 
and decision making will be similar enough that the 
experience can be transferred from the adminis ti.ative 
level to the teacher level. 

In planning the total teacher evaluation, consideration 
should be given to when certain activities should occur. 

Rome experience can be gained with the administrator 
evaluation plan, but the time when certain phases of 
the process will occur will be different. For example, 
the goal-setting stage or the final decision stages may 
occur in similar sequence to administrator evaluation 
but at somewhat different times of the year than with 
teachers . Other than legal constraints regarding contin- 
uation of contract, there appears to be no reason why 
considerable variation could not occur from district to 
district or from school to school . 

In addition to the sequence of events in the evaluation 
process, a choice must be made regarding whether the plan- 
ning will occur on a piecemeal or comprehensive basis. 

That is, will certain aspects of the evaluation process 
be considered independently of other parts and decisions 
made to change a given part; or will the entire teacher 
evaluation procedure be examined and nothing changed or 
implemented until plans are complete for the entire process? 
The following comments seem appropriate to this problem 
area: 



• The comprehensive approach is more likely to yield 
a well-integrated plan which satisfies the goals 
desired; however, it requires a mature staff and 

a spirit of cooperation and optimism regarding the 
approach. 

• The piecemeal approach risks early implementation 
of procedures which may not be compatible with 
desirable activities at a later point; however, 
it is often argued that any steps leading to 
professional growth of teachers will improve learning 
conditions for children . 
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CONCLUSION 



Planning teacher evaluation procedures consists 
of determining: (a) objectives y (h) people who 

will he involved^ (c) activities that should 
occur y and. (d) time sequence of events. The 
way decisions are made regarding this planning 
predetermine much of the effectiveness of the 
eventual procedures as well as the satisfaction 
with these procedures. 



Purposes of Teacher Evaluation 

One of the first steps in establishing or "revising a 
program of t eacher evaluation is the determination of 
purposes of the program. If the program is to be success- 
ful , it is necessary that these purposes be identified, 
discussed, and agreed upon by all who are involved in 
the process. It is important that all potential purposes 
be discussed openly and thoroughly; certain purposes 
should not be ignored just because there are critical 
issues involved in them or because their accomplishment 
may precipitate conflict . 

For example, one purpose for a program of teacher evalua- 
tion might be to provide a basis for sound administrative 
decisions regarding re-employment. Some people think that 
the accomplishment of this purpose creates an atmosphere 
which decreases cooperative relationships between admin- 
istrators; they prefer to ignore this purpose and hope 
that the problems inherent in re-employment will not 
arise. However, since even in small districts these prob- 
lems do arise, all purposes should be discussed openly and 
clarified in writing so that teachers and administrators 
may understand the bases for the evaluation program., 

NOTE 



Morale cannot be high if staff members are 
fearful or hostile. Since unknowns contribute 
to fear, evaluation programs which have written 
statements of purpose that are clear, precise, 
and complete are more likely to produce a sound 
basis for open communication and cooperatiiie 
relationships than programs designed around 
ambigu'’ is or unwritten purposes. 
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Purposes 
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Some 

Purposes 

Identified 



Inclusive 

Statement 



Purposes of teacher evaluation vary somewhat from district 
to district and include many of the following: 

• to improve teaching (including out- of- classroom 
activities as well as classroom instruction; 
this purpose is not limited to teacher behavior 
but implies any actions taken to improve teaching 
systems , the teaching environment , or teacher 
vehavior) 

• to supply information for modification of assign- 
ments (including placement in another position , 
reduction of load , promotion to a leadership 
position , or termination of employment) 

• to protect individuals or the school system 
(including both the protection of teachers 
against a capricious new administrator and the 
protection of the school district and children 
against a harmful teacher) 

• to reward superior performance 

• to validate the selection process 

• to provide a basis for career planning and indi- 
vidual growth and development of the teacher 
(including professional degrees and inservice 
training programs). 

All of these purposes might be expressed by saying: The 

purpose of teacher evaluation is to safeguard and improve 
the quality of instruction received by students . Implied 
in this statement is the view that evaluation of teachers 
will facilitate both teacher and administrator decisions 
(the teacher’s decisions regarding how to perform as a 
teacher and what will improve that performance, and the 
administrators ’ decisions regarding what actions he might 
take to aid students and teachers . ) 

The nature of the listing of purposes, or even of the 
presence or absence of a formal evaluation plan, should 
not alter the individual responsibility that teachers 
share with other professional practitioners for continuous 
self-improvement. However, if the school district has a 
well-organized progtam of formal evaluation with clearly 
specified purposes , it should be of assistance to the 
teacher in his voluntary program of continued self- 
assessment and improvement . 
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Changing 
Research 
V i ew 



More recently, the emphasis has shifted in the direction 
of attempting to determine why teachers vary their 
behavior from one teaching episode to the next and whether 
or not thi variation is itself associated with particular 
educational outcomes. The concern here is for variation 
of teaching behavior between visits and the consequences 
of this variation , 

For practitioners, the implication of this shift in 
emphasis is significant. The practitioner is interested 
in what works, and he realizes that some stable patterns 
of teacher behavior do not produce results in certain 
circumstances. He needs to know what produces I'^arning in 
particular, situations and how teachers can be encouraged 
to choose those behaviors which have a high probability of 
success , 



From a research standpoint, the purposes of evaluation of 
teacher effectiveness have changed somewhat in recent 
years. For a long time, the purpose was to show that 
certain types of teachers provide certain patterns of 
teaching. In order to do this, it v^as necessary to 
observe the teacher a large number of times to determine 
what his stable patterns of teaching were. 



Relation The teacher evaluation program should not ignore other 

to District aspects of the school program. The function of evaluation 

Goals is to facilitate the accomplishment of the goals of the 

organization; therefore , the purposes of evaluation should 
be established following a complete review of the goals of 
the school district. When this has been done, the purposes 
of evaluation are more likely to be compatible with and 
contribute to the school district goals. 

For example , lO reward superior performance can be a legi- 
timate purpose of evaluation, but it should be clearly 
related to the goals of th^ organization if it is to be 
included in the purposes of evaluation. If it is clear 
that rewarding superior performance provides incentive 
and mo tivation for creative teacher behavior which causes 
children to learn better, then to reward superior per- 
formance seems a reasonable purpose for evaluation. 

However, some may desire to initiate a reward system for 
the purpose of controlling the behavior of teachers 
(without regard for the effect on accomplishment of educa- 
tional goals) and this may lead to a misuse of the evalua- 
tion function , 
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CONCLUSION 



Possible 
Adverse 
L cf ects 



Human 

Relations 

Tensions 



Reduction of 
Creativity 



Reduction of 
Assistance 



The purposes of teacher evaluation progrcvns 
should grow out of clearly stated goals of 
the school system and should coy'itrihute to 
the accomplishment of these goals. 



Possible Problems in Teacher Evaluation 



Adverse effects are indications of problems which exist 
and their examination may indicate the source of a problem 
Some of the possible adverse effects of teacher evaluation 
include : 

• human relations tensions when poor performance is 
perceived by the evaluator 

• reduction of creativity it evaluation system is 
too rigid 

• reduction of assistance provided by supervisor or 
principal 

• poor validation of selection process. 

Evaluators need to be aware of the fact that diagnosis of 
the teaching act ^ primarily cognitive , i.e,, diagnosis 
primarily involves knowledge. However, interpretation of 
this knowledge by the person being evaluated is not 
necessarily cognitive at all, since it may be interpreted 
on an emotional level. When evaluation procedures 
include placing people in categories, e,g,, ’’good" or 
** aver age , " an emotional response is quite likely to be 
precipitated on the part of some teachers. Regardless of 
the evaluation program, evaluators should be prepared for 
emotional responses and should provide psychological 
support for the teachers being evaluated. 

Creative teachers can function in a variety of situations. 
However , teachers do tend to be affected by the form of 
evaluation used. If the evaluation criteria require 
strict adherence to predetermined procedures and materials 
the constraints are likely to reduce the motivation for 
divergence and creativity . 

When the principal or supervisor has both the responsi- 
bility of helping the teacher improve and the job of being 
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Selection 

Validation 



accountable to the school syc^tem for results, he some- 
times has the tendency to emphasize accountability to 
the extent that it adversely affects his ability to help 
the teacher. At other times, his ability to assist the 
teacher is reduced simply becr.use he has the responsibility 
of accountability. Some consider this reduction to be a 
result of a natural distrust which teachers have of the 
use to be made of information obtained through the eval- 
uation process. However, it may be due more to lack of 
knowledge of how evaluation _is_ used and must be used by 
personnel who are charged with responsibilities to help 
teachers, viz., the evaluation information forms a basis 
for the counsel and assistance provided. Without it the 
assistance would be shallow and meaningless. The issue 
is not whether those who supervise and provide assistance 
should evaluate teachers; rather, it is how information 
obtained through the evaluation process should be used. 



A number of reasons exist for poor validation of selection 
procedures , However , foremost among them is the fact that 
very few school districts have reliable measures of teacher 
performance , Suppose , for example , that a principal is 
concerned with the fact that teachers will not accept him 
as a person who can help them if he emphasizes accounta- 
bility to the system and attempts to measure their behavior 
and performance accurately, I'/hat is he likely to do and 
how will this affect the validation of selection procedures? 



Under tliese circumstances , a principal is likely to eval- 
uate all teachers more favorably than he should , indica- 
ting more uniformity than exists. In effect, his evaluation 
reflects a rather general "halo effect" and the measures 
approach a constant, Wlien this occurs, no predictor can 
be found that vjill predict teacher effectiveness for that 
situation , Since all teachers appear to look alike once 
they are evaluateu on the job, any predictor variable will 
predict as effectively (or more precisely, as ineffectively) 
as any other , * 

The same consequences result if an evaluator allows his 
measurements to migrate toward a central tendency. Like- 
wise, if the measurements provided by an evaluator are in 



*An ej.planation of what happens in the correlation computa- 
tion when one of the variables approaches a constant is 
provided in Appendix 1.1, 
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in fact not related to the behavior of the teacher or the 
results of this behavior, then the correlation with any 
predictor variable is likely to be zero. In affect, any 
evaluations which approach some constant value or which 
approach some random assignment are likely to be worth- 
less for validation of selection procedures. 

Many of the problems underlying the adverse effect mentioned 
above are directly traceable to poor measurement procedures. 
Problems of measurement include the following: 

• prejudice, bias, or poor j udgment 

• inconsistency of reaction to behavior 

• subj ective ratings and classifications (or classi- 
fication schemes which require high inferences to 
be made) 

• influence of the personality of the teacher outside 
the classroom on measurement of behavior in the 
classroom 

o attempt to measure too many elements of classroom 

situations (especially at the beginning of a program) 

• tendency to continue a prior viewpoint of a personas 
performance 

» consistent overvaluation or undervaluation. 

The last problem, viz . , overvaluation or undervaluation, 
deserves additional consideration. Overvaluation sometimes 
is caused by a desire to avoid antagonisms with teachers, 
an effort to make the supervisor look good, or a sympathetic 
response to a certain individual’s problems (e.g. , age, or 
problems outside the school situation . The desire to avoid 
antagonisms is a strong motivator for some principals and 
supervisors, especially those who see themselves as place- 
bound, desiring to live harmoniously with teachers over a 
long period of time, and being fearful that accurate 
reporting of measurements will damage the harmonious 
relationship . 

However, the practice of indicating to a teacher that he is 
performing well when in fact be is not (i,e., of continually 
overvaluating his performance) can be as much a disadvantage 
to him as it is to the organization. The reason for this 
is that the teacher then has a limited basis for improve- 
ment in comparison with what he would have if full informa- 
tion were provided. Of course, the school organization and 
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Sometimes it is difficult to determine wliether poor 
result.-- are due to pour performance on the part of tlu* 
teaclier or to situational constraints tliat prevent better 
teacher performance. Wien there is any doubt on tins 
question, it is essential that the conditions under which 
the teacher works receive direct attention in making an 
evaluation cf the teacher's effectiveness. Sometimes the 
environment (including supplies and materials as well as 
the psychological environment) is such that learning is 
inhibited, and a review of those situational factors that 
might be expected to influence the effectiveness of the 
teacher will help to prevent poor judgments regarding the 
teacher . 



Another problem of teacher evaluation which should be 
considered when making plans for goal-setting conferences 
v;ith teachers is the feasibility of evaluating all teachers 
annually. Many school districts do not have an adequate 
administrative and supervisory staff to do an adequate job 
of evaluation . In analyzing this problem in local school 
dniitricts, there should be an attempt to relate the problem 
to purposes for which the evaluation program is designed. 
However, if one of these purposes is to assist teachers in 
improving their instruction, it becomes very difficult to 
justify a program which does not include continuous evalua- 
tion of every teacher. 



Consider an analogy to a baseball team. It is inconceiv- 
able that any major league team would discontinue observing 
the production and actions of a given player after he had 
been on the team for three or five years. This raises 
the question of why batting coaches still v;atch hitters. 

It seems reasonable to conclude that they v;atch batters to 
determine whether they are still hitting the ball , under 
what conditions they are hitting the ball, and whether 
there appear to be any flaws which hinder improved hitting. 

It also raises the question of the number of coaches needed, 
assuming that coaching (rather than umpiring) is desired. 
When Mickey Mantle joined the coaching staff of the New 
York Yankees in September of 1970, he became the fifth 
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coach in addition to the field manager to work with 25 
professional ball players. How many coaches are needed 
for an athletic team of x players, i.e., what is the 
person/coach ratio?* 

NOTE 



If part of the function of the evaluator is 
to provide assistance for teachers (i^e.^ 
to be a coach rather than an umpire) ^ then 
consideration should be given to the number 
of evaluators needed to do an adequate job 
and efforts should be made to work toward 
acquisition of these evaluators. 



Desirable Outcomes and Evidence of Them 



What are the outcomes that should be sought in a goal 
setting conference? Certainly the two major items which 
should be considered are: what behaviors are important 

and what should result as a consequence of these behaviors. 
Most school districts expect different results from differ- 
ent teachers, since grade levels and subject matter 
differences anticipate differing expectations. Yet, 
most school districts use the same evaluation procedures 
and behavioral expectations (at least as judged by 
uniform evaluation forms and observation guides) for all 
teachers . 

NOTE 



It seems reasonable for a school system to 
expect certain desirable behaviors from all 
teachers in the system. However^ it is also 
desirable to have different sets of expecta- 
tions for the evaluation of different groups 
of teachers^ the expectations varying accor- 
ding to the tasks set for each group. 



^People in policy making positions might raise this ques- 
tion in comparing the student/coach ration on the high 
school athletic teams with the teacher/ supervisor ratio 
on the high school staff. 
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For example, consider lln*oc factors which miplu be used 
to describe teacher behavior,- One migiit be called an 
F.inpathy factor and would describe behavior ranging from 
’’warm, understanding, and friendly” to ’’aloof, egocentric, 
and restricted,” A second factor might.be Buoy ancy and 
would describe behavior ranging from ’’stimulating, 
imaginative, surgent” to ’’dull and routine,” A third 
factor might be Organization , describing behavior ranging 
from "responsible, businesslike, and systematic” to 
"evading, unplanned, and slipshod,” 

The three factors are illustrated in Figure 1,1, Points 
A, 13, and C in the three-dimensional space represent 
behaviors of three teachers. Teacher A exhibits low 
Empathy, Buoyancy midway between high and low, and high 
Organization; Teacher B exhibits Empathy and Organization 
midway between high and lew, and high Buoyancy; Teacher C 
exhibits low Empathy , Buoyancy , and Organ izat ion , 




BUOYANCY 



Figure 1,1, A Representation of Three Factors 
of Teacher Behavior, 



-The discussion in this section draws from the work cf 
Ryans (1960) , 
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The question which might be asked of Teachers A and B is 
whether these behaviors could be equally successful in 
the same situation. Likewise, would different situa- 
tions indicate that one of the two teachers would be more 
successful than the other? Some teachers can shift from 
position A to position B, while others are less adaptable 
in their behavior. Some teachers appear to be at point C 
and have difficulty in changing their behavior. 

Although the ^'eacher behaviors which are beneficial for 
one group of children might not produce the same results 
with another, there are some teacher behaviors which 
have precipitated desirable pupil outcomes in a variety 
of situations. Students seem to profit from a teacher 
who: 



e accepts and uses ideas and opinions of pupils 

• is flexible and adjusts behavior and strategies 
to situations and students 

• views teaching as a complex task which requires 
goal setting , individual student assessment, and 
decision making in terms of immediate and long- 
range problems 

• provides students with a framework within which 
to interpret information. 

In discussing teacher behavior, it is desirable to empha- 
size descriptive terms prior to attaching any value to 
these descriptions. Likewise, it is wise to avoid terms 
that are emotion laden, such as ^democratic’* or ’’progres- 
sive.** 

In addition tc discussing performance (in terms of behavior 
and/or output) in the goal-setting conference, at least 
one other item should be considered: r.he potential of 

the teacher for additional tasks. Since this is directly 
related to the purpose of providing career counseling for 
teachers, it should be included in the discussion of goals 
to be sought. 



Observing Teachers and Collecting Information 

The tools and techniques that one uses to collect informa- 
tion for evaluating teachers must be related to purposes. 
Since multiple purposes usually exist, one question which 
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arises is whether the system for collecting data ade- 
quately serves all of the purposes. For example, will 
the techniques and procedures used to accomplish the 
purpose of improvement of instruction serve all other 
purposes? 

SUGGESTION 



Fooun on developing a system of information 
collection which satisfies what is conoidererl 
to be the major purpose of teacher evaluation. 
Then examine the system developed to determine 
if adjustments are needed to serve all other 
purposes. 



In making plans to collect information for the evaluation 
of teachers, the following questions should be considered: 

• k^iere will the information be acquired? 

• !^Tlat will the information look like? 

• Hox\^ will a sample of total information be acquired? 

o Vlho will collect the information? 

• How much training will be required to collect it? 

Sources of Information 

Three basic sources of information are available for analy- 
sis and interpretation prior to evaluating teachers; 

• in-classroom behavior of the teac her, as perceived 
by students, the teacher being evaluated, other 
teachers, administrators or supervisors, para- 
professionals such as teacher aides. 

• out-of-classroom teacher behavior , as perceived 
by students, the teacher, other teachers, admin- 
istrators or supervisors, teacher aides, and 
other paraprof essionals such as cooks and cus- 
todians . 

• student accomplishment , as measured by teacher- 
made tests, standardized tests (achievement, 
attitude, or skill performance), student 





Choices 
to be 
Made 



Need to 
Reduce Data 



self-report devices, observations of student 
behavior (by teacher, outside observer, parent), 
student products or projects. 



Nature of Information 



Since teacher evaluation is dependent upon measurement 
in gathering information, care should be taken to develop 
procedures and train people so that appropriate instru- 
ments are chosen and used effectively, 

CAUTION 



Instruments should not he chosen solely on 
the basis of the evaluator's familiarity 
with the instrument, its availability, or 
the fact that other districts are using it. 
Consideration should include: 

p relevance to goals 

9 acceptahility by those who are involved 
9 accessihility of information 
9 time needed to acquire information 
9 cost. 



In order to evaluate teachers effectively, the informa- 
tion which is available must be reduced from its raw form 
to a form which may be analyzed and interpreted. Fo?.‘ 
example, "raw" data regarding the teacher behavior might 
be either the behavior itself or an audio or television 
tape recording of classroom activities. When "reduced" 
it might take the form of a matrix of figures of comments 
by an observer. Raw information regarding student accom- 
plishment might be the answer sheets of an achievement 
test, while the reduced data might be a letter grade or 
rank in class. 

NOTE 



The procedure used for r>educing data from 
its raw form influences the final inter- 
pretation of the information collected. 
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Rc‘garcl.l,os8 oC whether one is concerned wiLh Leachc'r 
behavior or results of this behavior (c.g., sLudouL 
accomplishment), when the raw data is reduced it pcMi- 
eraJ ly takes one of the following forms: 

• rank order 

• forced distribution 



• absolute categories 

• verbal descriptors . 



Rank ordering is simply the ranking of individuals in a 
group according to some item or characteristic. A forced 
distribution requires that a certain percentage of the 
item being considered be placed in each descriptive 
category. For example, one might require that individuals 
be placed in five categories according to the following 
ratios: 10%, 20%, 40%, 20%, 10%. Absolute category 

systems describe individual behaviors or total behavior 
in a classroom by placing incidents or time periods into 
discrete descriptive categories. For example, Flanders’ 
Interaction Analysis system is designed to categorize 
vergal interaction. Verbal descriptors are used by 
observers to express what has been perceived. They may 
be in sentence or phrase form and depend on the behavior 
exhibited and the facility of the observer to describe 
the behavior in accurate terms. 




Reducing the raw data implies that some type of measurement 
is occurring, and measurement itself implies a category or 
numerical system that is precise. But having a category 
system or a numerical system does not imply that the 
system is precise, since the use of the system determines 
the precision. 



For example, if a person were asked to count the number of 
shots taken in a basketball game and record approximately 
where they were taken and who made them, he might be able 
to do this xs^ith extreme accuracy. However, if the same 
person were asked to determine how many times a hockey 
player was out of position during a game, the result 
might be quite inaccurate . He might record a number that 
looks very precise but that may deceive. 
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CAUTION 



Systematic 

Observation 

Procedures 



A wrong number is wrong even if it lookr, 
very precise. As much care should he taken 
in designing the implementation and use of 
a measurement system as in the design of 
the system itself. 



Observation of classroom behavior . Recent developments 
in classroom observation techniques has increased sig- 
nificantly the number and type of observation guides 
available. Simon and Boyer (1970) describe 79 different 
observation schedules available for use; many have been 
used for research only, but are quite easily adapted for 
evaluation purposes. Some measure very special aspects 
of classroom behavior (e,g,, verbal interactions), where- 
as other instruments are broader in scope. 

There have been two major applications of classroom inter- 
action analysis procedures : 

(B to help an individual develop and control his 
teaching behavior 

• to discover how to explain the chain of events 
which occur in the classroom. 

It seems obvious that either of these applications could 
be of use in evaluating teachers . 

Although there are many observation guides available and 
although many of them could be useful in evaluating 
teachers, their use is not as widespread as less precise 
measures. The reasons for this lack of use appear to be: 

• information has not been broadly disseminatc-.l 

o very few opportunities have been provided for 
learning the skills necessary to use the 
techniques . 

There are some notable exceptions to the limited use of 
systematic observation procedures . For example , the use 
of Flanders’ Interaction Analysis system has increased 
dramatically in the Northwest because of workshops con- 
ducted by the Northwest Regional Educational Laboratory 
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should be mentioned: 

• l-Jlien too many ratings are clustered at a particular 
point , the inference is that raters are overly 
lenient , too harsh , or are unwilling to be 
decisive and objective. 

• It is easier to identify the very poor or the 

very good than it is to differentiate in the middle 
range of a rating scale . Therefore , decisions 
concerning middle range ratings are more difficult 
to justify. 

SUGGESTION 



If the]j cn?e used in eva'luath'ig teachers^ 
the aocuraQij of rating soales or check- 
lists fmy be improved by: 

0 clearly d.efini'njg the focus of the evaluation 

0 developing specific^ low-inference itens 

0 using a cormvon record, form 

0 providing adequate training for observers 



‘-Probably the best single source for describing the types 
of analytical and general observation procedures being used 
in public schools is Evaluating Teaching Performance (1969). 
(See Bibliography. ) 
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Measuring Out-of -Classroom Behavior . To the ex ton i. th.it. 
the activities of the teacher Jn roles other than class- 
room instruction are considered to be important as i part 
of the assigned or expected responsibilities, clocunuml at y 
evidence should be assembled regarding these activities. 

As with classroom behaviors, the expectations of the school 
organization should be discussed with the teacher ctiul hc‘ 
should have an opportunity to set goals that are compatible 
with his own interests and ability to contribute. 



Since out--of-otassvoom activities ave sometines 
sources of conflict (or at least subject to 
differences of interpretation) between teachers 
and the community^ new teachers should be 
oriented to local custo?vs and expectations. 



In measuring out-of-classroom behavior of teachers, 
advantages and disadvantages exist. The advantages seem 
to be: 

9 information to be obtained is amole, often 
pertinent to the job expectations 

• information is relatively easy to ob tain , 

The disadvantages appear to be: 

9 reliability ' of information is difficult to check 
9 sampling information is difficult and limited 

• information obtained is difficult to score or 
interpret , 



Measuring Student Accomplishment . Measures of pupil ouL- 
comes include how pupils think, perform, and feel. 



NOTE 



SUGGESTION 



Written records of ont-of -classroom behavior 
should he kept and discussed regularly with 
teachers in order to check the reliability 
and meaning of the information 
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j r. :'l i I i • m; 1 1 iiu asurt’S include: 

9 {<n^n-;!cdge and ahilicy rncasaircs (whal: a per.s(^f{ 
knows ) 

• skill perftuananco measures (whan a person can 

( 1 (^ ) 

9 a U:i nucl Ina.l measures (whan a person feels or 
des Ires ) 

0 innet'osn measures. 

Sv:liO(3ls are established to facilitate pupil J earning; 
therefore, the ultimate criterion for teacher success is 
the amount of learning that occurs in pupils for whom he 
is responsible. Thus, one of the advantages of measuring 
puj:>il growth is that it is a direct measure of outcomes 
that are desired. However, there are a number of dis- 
ad vantages . 

}h)r one thing, there is immediate grovjth and there is 
long-term growth. The behavior of a teacher may not con- 
tribute much to the immediate learning of a pupil but may 
have a long-range effect on attitudes and behavior. Or 
conversely, short-term gains may be lost in the long run, 
or short-term academic gains may be obtained at the 
expc.nse of long-term negative attitudes which prevent 
later learning. 

Another disadvantage is the difficulty of adequateJ.y con- 
trolling the situation so that growth can be attributed to 
tlic behavior of a given teacher rather than to a v/ide 
range of uncontrolled conditions which impinge on 1. earn- 
ing. Because of this problem, consideration should be 
given to flie effect of a succession of teachers on a 
pupil achievement. This should not be done in lieu of 
examining t:he student achievement obtained as a result of 
individual teacliers but should be in addition to this 
ana lys is . 

Historically, student accomplishment has been avoided as 
a means for evaluating teachers, primarily because of 
tile difficulties involved. However, recent emphases on 
accountability of teachers for productivity have caused 
rcuevjed interest in setting specific student accomplish- 
ment goals and attempting to attain them. If the trend 
continues, It is quite likely to have a significant 
impact on how teachers are evaluated. 
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Self-Evaluation . Teacher self-evaluation may he concerned 
with either classroom behavior, ou t-of-classr(K^m behavior, 
or student accomplishment. It differs from other evalua- 
tion of teacher effectiveness in that: 

• There is no need for involving an exLcrnai 
observer in the measurement process. 

• Behavioral criteria are determined by the 
teacher rather than by some external source. 

The advantage of self-evaluation seems clear : the 

teacher has the opportunity for improvement without ex- 
ternal threat. The primary disadvantage is that the 
standards used for evaluation may not relate readily to 
outside criteria or needs of the school district . 

Before implementing a teacher self-evaluation program, 
a school district should : 

• provide teachers with training to help them 
specify their own goals in measurement terms 

• provide teachers with a framework (e.g., an 
observational system) for analyzing and inter- 
preting their o\m behavior 

• provide teachers with technical competence 
needed for operating the various new media 
used for recording their own behavior. 



Self evaluation veduoes the threat of out- 
side intervention and^ therefore has potential 
for increasing motivation and creativity » How- 
everj it may not serve all purposes of evalu- 
ation^ since external standards may be ignored 
and administrative decisions may not he facil- 
itated. 



Sampling Information 



NOTE 



How Often 
to Observe 



How often should a principal observe a given teacher? A 
quick and obvious response is **Moi*e than most principals 
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(>iiSL'rvc‘ LoacliLM\s , hut this ^ivus very .llLt.le assistance 
ic one seeking girido.l i.ncs . A more her.eflcial rosp<msc 
to this qiiesLi.on depends on a number of factors, inc' hiding, 
the purposes for the observation and the resources 
available. To satisfy most purposes, each teaclu'r should 
be observed In the classroom environment several Limes 
annually at different times of the day or in varv i nu 
types of instruction. Such observations can be a mixtun' 
of observation procedures and may be made by appointment 
or not. 

How to Consider the following as a means for determining the 

Occidc amount of classroom observation to be done by a princi- 

pa L : 

o Specify the amount of time per week which should 
be spent in teacher evaluation procedures. 

o Ind i ca te how long ea ch observation should he and 
whether the observation should be folloxv^ed by a 
teacher conference and written record . 

o Dec ermine the approximate number of teachers who 
can be observed per week, on the average. 

o Determine hovj many times each teacher should be 
observed annu;\ily. 



An example Let us use an example to see what would happen if the 

abuve procedure were follow^ed. Suppose that v/e create a 
hypothetical situation in v/hich: 

• The principal should spend 5-8 hours per week in 
t eacher evaluation procedures . 

© l.cngth of classroom observation is approximately 
30 minutes. 

• Observation will always be followed by a teacher 
conference . 

o Wr i tten records will be kept of observations and 
conferences. 

© There are 60 teachers in the school. 



*In an Nl^A survey (1969), 80% of the responding schools 
indicated that they evaluate probationary teachers more 
often than tenure teachers; yet, 85% of the probationary 
teachers were evaluated no more than twice annuallv. 
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For this example, the principal would be able to observe 
approximately five teachers weekly, on the average; this 
would allow him to make a systematic observation of each 
teacher approximately every 12 weeks or only tliree times 
annually. 

Is this sufficient for accomplishing the purposes estab- 
lished for teacher evaluation in this district? If not, 
then steps should be taken to either: 

• establish more reasonable goals for the evalua- 
tion program, or 

• modify the procedures established (e.g., spend 
more time in observation weekly), or 

• acquire more resources for doing the task. 

If more assistance is needed, then the principal and the 
personnel in the central office should agree on who and 
what the nature of the assistance should be. For example, 
should the vice principal, department heads, consultants, 
and central office personnel be involved? If so, should 
their role in the total evaluation process be different 
from that of the principal? 

NOTE 



Some teacher evaluation programs are designed 
for failure because not enough personnel are 
provided to do the Job adequately. Personnel 
inadequo.cies should be relatively easy to 
identify^ and a number of alternatives may 
exist for remedying the problem. 



Wlio collects information ? To evaluate teachers, informa- 
tion should be acquired regarding classroom behavior, 
out-of-classroom behavior, and student performance. Wlio 
collects this information depends on the plan which is 
implemented, and the following people should be considered 
when developing a plan: 

• principals 

• vice principals 
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• lU'p.'uM ru'H I lic.uis 

• suh'M'r.L inaf:l:er sihh' i a 1 i s Ls (consulLanL s , sunc'r- 
V i. s o r s ) 

• ^iencral consultants 

• personnel spec Lai i sis or directors 

e peers (otl)er Leacliers) 

e students 

• parentis 

It seems reasonab.lc to expect that different people v.’ouJd 
^.’olJect different types of information. For example, one 
would expect that a principal of a large liigh scliool v^ould 
collect different types of information from that of the 
deiwirtment head or the subject matter specialist. Because* 
tiie difficulty of liaving expert knowledge in all of 
tile subject matter specialties of the various teachers, 
the principal is more likely to provide meaningful feed- 
back to teachers in the areas of classroom atmosphere, 
general learning principles, and verbal and nonverbal 
i n tcraction . 

Th.e subjtH't matter specialist or department head, on the 
c'>tiier hand, could collect information concerning the 
appropriateness of the content and activities related to 
the content. The feedback that he is able to provide to 
teachers regarding content and activities that are a]?pro- 
priate for the given subject specialty should he of bene- 
fit to teachers. The judgments made about these aspects 
of teaching are part of the specialty of the consultant 
or supervisor, who should be made to feel a part of the 
process of teacher evaluation as much as other people 
who colJoct and analyze information regarding the teacher. 

nifficultios arise with regard to use of information 
collected by subject matter specialist*:.. Some seem to 
fear being labeled a '^tattler’’ or to feel that the task 
of collecting information for evaluation purposes is 
outside their role. A clear specification of the role of 
the subject matter specialist and of the use of the 
inforaation he collects should help to remove some of 
tliesc' di ff icuJ ties. This specification is the job of 
th(i administrator, who has a responsibility to set objec- 
tives that .help to harmonize the activities of people 
vn* th the goals of the organization. 
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Prerequi- 
sites to 
Good Obser- 
vation 



Limits to 
Observation 



Training required for observation and information collec- 
tion . Training teachers and supervisors in the use of 
an information collection system will assist in providing 
a common language for analyzing and discussing the 
teaching-learning process , for examining the contribution 
that out-of-classroom activities make to the school 
program, and for determining student accomplishments . 

Most information is collected by means of observation, 
and good observation has certain prerequisites ; 

• Some purpose needs to be identified. A person 
does not just observe, he observes for some- 
thing. He does not just look but he looks for 
something specific . 

• The more specifically one identifies what he 
is looking for, and the more systematically 
he plans for observation, the more likely it 
is that he will know something following the 
observation. 

• What is seen needs to be systematically recorded 
and organized in relation to purposes. 

• What is observed needs to be subj ect to checks 
and controls , in order that some determination 
can be made of the validity, reliability, and 
precision of the observation. 

Observation is direct. It is not an indirect predictor 
of behavior as a test is, but rather tells something 
directly about the behavior of teachers . As a result 
of this directness, it has some limits . For example , 
some events cannot be predicted and, therefore, are 
difficult to observe. A teacher who has a very sensitive 
ability to handle delicate human relations problems in 
the classroom may not be able to demonstrate this for 
an observer because of the difficulty of predicting when 
an appropriate situation will occur. Likewise, the dura- 
tion of events is a very practical limitation to observa- 
tion of classroom events. It sometimes is hard for an 
observer to see the continuity and sequences of events 
that make some teachers extremely effective. 

Once the prerequisites to good observation have been 
provided, what about the act of observing? The observer’s 
mental set during observation is quite important. Other- 
wise, he cannot interpret gestures , expressions , etc. 



59 



72 



This means Uiat lie nuis L know someLliing about tlic con Lex L 
WLtlii.n wliLdi lie is observing, and implies Lliat lie sliouM 



• discuss the situation whicli he will oliserve 
with Llic t Gaelic r prior to observing 

• confer with the teacher following the observa- 
tion to check his own understanding of the 
context 

• develop his o\m understanding of the impact of 
contexts on both students and teachers . 

An observer *s introspection and experience can he both a 
hindrance and a help. It can cause him to overlook and 
misinterpret as well as to be very shrewd in perceiving 
subtle differences. 



Obsevvevs of teacher heJixxvzov and classroom 
interaction should develop means for checkinc 
their reliability; this usually means 
comparing observations with another observer^ 
and the teacher being observed.. 



Because an observer is usually responsible for knowing 
something about the situational context, the interaction 
-'■imong people , and individual behavior , he should attempt 
to increase reliability of observation by; 

® adequately defining what is to be observed 

• exaTTiining his own background and experience 
to determine whether it might be distorting 
his perception 

0 establishing categories which assist in 
recording behavior 

• comparing observations with others to help 
establish and maintain reliability . 

Summary: Collection of Information 

Collection of information implies the use of a measure- 
ment procedure. Because of this, one might reasonably 
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revise the question of how to develop the characteristics 
of a good measurement device for collecting information 
to evaluate teachers . 

The three major characteristics of a good measurement 
device are validity, usability, and reliability. To 
acquire validity , it is necessary to decide what is 
important in a given situation. To develop a usable plan 
for collecting information, it is necessary to make it as 
simple and streamlined as possible. To develop reliability, 
it is necessary to train those who are involved in collec- 
tion of information. Likewise, training is required to 
obtain validity and usability. 

Some guidelines for development of information collection 
procedures for use in evaluating teachers include : 

• School board policy should identify all purposes 
of evaluation, specify general procedures to be 
used in evaluation of personnel, and identify 
what types of data the school board desires in 
order to provide information to the public and 
to set policy. 

• All new teachers should be oriented to the total 
procedures used, as well as the forms and reports 
that will be used. 

9 Vice principals and department chairmen should be 
involved in evaluation procedures, and all persons 
involved should be acquainted with their role. 

9 Student reactions to teacher behavior and to 
classroom activities should be encouraged on an 
anonymous basis at the option of the teacher. 

9 If peer evaluation is used, training in making 
honest and helpful comments should be provided. 

Post-Observation Conferences , Communication 

In the same manner that clear and precise communication 
is essential to establishment of acceptable purposes for 
teacher evaluation , excellent communication between 
teachers and evaluators is essential in the post-observation 
conf erences and formal reports . Every classroom observation 
should be followed by a discussion between the teacher 
and the cooperating evaluator (s) , and this discussion 
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slum Id Lake place as soon aft:er Lhc ohscrvaLlon an 
possLbU.*. There should be open communicaL Ion ree;aiul i up, 
what will, be reported to tlie central office, and writL^m 
copies of any report sliouid be given to tlie teacher. 



When l:n 
Make I'os t- 
observat ion 
hoc i s i cms 



S.ince one of the functions of tlic total eva.luation proctess 
is to make decisions possible, one of tlie first consicU'r- 
ations an evaluator must face is when he will make the 
decisions that are based on observations. He lias three 
choices; he can make the decisions: 



• before the post-observation conference, allowing 
him to use the conf 3rence for informing the 
teacher of the decision or 

9 during the conference , allowing the teacher to 
be a part of the decision process or 

• following the conference, using the conference 

as an opportunity to colie ct additional information 
that might be pertinent to the decision. 



Some practitioners favor one or the other of these times 
for all decisions, but others tend to make a choice in 
terms of the purpose to be accomplished by the particular 
conference or decision. For example, if a certain decision 
needs to be made regarding classroom procedures and the 
evaluator is providing information to be of assistance to 
the teacher, the decision might be made during the con- 
ference. Hovjever, if a decision is to be reached reearding 
retention of the teacher, the evaluator may delay the 
decision until following the conference in order to use 
the information obtained from the conference (or possibly 
from other sources) to make the final, decision. 

Hesearch Researcli on the use of post-observation conferences indi- 

Results cates that the following statements are warranted: 

9 Criticism has a negative effect on employees; it 
tends to build defensiveness. 

9 Praise !ias very little effect on future productivity. 

o Mutual goal-setting for the future improves 
performance. 

9 Assistance and coaching effects better results 
when it is done daily rather than once yearly. 
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Developing a 
Plan for 
Conferences 



Sources of 
Conflict 



• Teachers accept decisions more readily if tlie focus 
is on improving performance and the situation. 

9 The number of improvements that can be accomplished 
at any one time is limited; therefore, one should 
choose a few and focus on them. (This probably 
implies the need to develop a specific strategy 
for assisting teachers.) 

To develop a workable plan for post-observation conferences, 
the following suggestions should be considered : 

• Provide a written guide to aid principals and super- 
visors in conducting post-observation conf erences . 

• Provide demonstrations and conduct practice sessions. 

• Discuss how the f ol lowing topics and activities 
can be included in the post-observation conference: 

— purpose of the interview 

— description of favorable information (feedback 
should be honest rather than effusive praise) 

— discussion of weak aspects of performance 
(constructive criticism must be given in 
friendly , cooperative spirit) 

. — asking for reactions 

— responding , discussing (help the teacher to 
know how well he is doing and what is expected) 

— considering appropriate action with teacher 

— determining what additional information is needed, 
when it will be sought 

— planning the next steps 

— concluding the interview 

The primary problem with the follow-up conference is that 
it can evolve into a conflict situation which creates a 
gip between administrator and classroom teacher. This is 
especially true if: 
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If an administrator or supervisor is concerned with the 
possibility that a conflict situation might arise and 
desires to take steps that might reduce the gap between 
‘nimself and a classroom teacher, then he should: 

o estal)lish open, authentic communication with teachers 

e seek agreement between the goals of the school sys- 
tem and the individual aspirations of teachers 

• be v/illing to be evaluated by teachers on those 
aspects of his job whicli affect the teachers . 

Especially pertinent to the open communication between 
administrators and teachers is the necessity to maintain 
open files of formal v/ritten evaluations. EJhen teachers 
know what is w^ritten regarding tl^.eir performance and 
what is reported to the board of education, anxiety and 
rumor are reduced.* 



Assessment of the Teacher Evaluation Process 

Because evaluation of teachers is an evolving process, 
the policy adopted by any local board of education should 
include plans for the periodic review of the evaluation 
program. Since the plan for e'^^a'I nation should be compre- 
hensive enough to examine the entire evaluation program, 
the means for gathering information about the effectiveness 
of the evaluation process should be planned at the same 
time the evaluation procedures are planned. 



*A short analysis of authentic 
as i t may relate to open files 



1 . 2 . 



and phony communication, 
is provided in Appendix 
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Analysis of 
Problems 



Use of Exit 
Interviews 



Integrated 

Personnel 

Program 



An analysis of the difficulties in the total evaluation 
process includes the necessity to examine the realism of 
the goals of the process, the effectiveness of the teach- 
ing procedures, and the adequacy of implementing the 
procedures decided upon. In examining all of these 
aspects of the evaluation process, one should seek answers 
to the following questions: 

• Is the instruction improving? 

• Are teachers receiving assistance? 

• Are students learning? 

• Are teachers with problems improving? 

• Are consistently ineffective teachers being 
selected out ? 

• Do teachers understand what is expected of 
them? 

• Is adequate information being provided to be 
able to improve the selection process? 

o Is the board of education provided adequate 
information for making personnel and policy 
decisions? 



Some of these questions may be answered by obtaining 
answer*') from teachers or administrators, students, or 
parents. Others may be answered by interviewing those 
teachers who either resign or are released. Businesses 
and industry use the exit interview very effectively to 
acquire information regarding : 

• why individuals leave the organization 

• employee ’s perceptions of problems in super- 
vision and evaluation 

• employee’ s perceptions of problems in selection 
and placement procedures . 

One aspect of the evaluation procedure which should be 
assessed is how well it is functioning with respect to 
other pex^sonnel tasks such as selection, supervision, and 
inservice training. If a problem exists, one procedure 
which may help to integrate all of these functions is to 
involve principals and supervisors in all of the personnel 
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runctions menLioned, viz , , selection, evaluation, suj'K^r- 
vis.ioii, and inservice training programs, 

ir/iiniiig Other prob.lems of teaciier evaluation may not be so.lved 

I'.valuators easily, and additional training may be needed. Iwen the 

best principals and supervisors may need to be trained 
to avoid allowing their personal biases and prejudices to 
affect the accuracy of observations. Tliey may need to he 
trained in using observation procedures wliich use defini- 
tions of behaviors to be observed and standards to be 
applied to each behavior . 

Public school systems treat the training of evaluators 
much more casually than industry does, in spite of evi- 
dence that training is likely to increase: 

o validity of decisions 

• reliability or consistency of decisions 

o discrimination of measurement 

« feelings of certainty regarding decisions. 

Procedures which are used by many businesses and industry 
and some school districts for improving the performance 
of evaluators include ; 

• elective inservice courses 
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o written documents or manuals 
G individual consultation 

The assessment of the teacher evaluation process should 
involve adequate planning for ways of gathering informa- 
tion, asking questions that are pertinent to the purposes 
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of the program, use of exit interviews as well as direct 
observation, and use of training procedures to assist in 
eliminating errors • 



Starting a Comprehensive Personnel Program 

This chapter has presented some views of current practice 
and research results regarding the selection and evalua- 
tion of teachers. Many of the comments have been infer- 
ences obtained from either research or practice, and 
there has been an attempt to write the ideas so that they 
would be applicable to practitioners. 

Approach to The approach for the evaluation of teachers assumed the 
Evaluation need to develop evaluators who are accurate observers of 

behaviors that teachers, supervisors, and administrators 
consider important. The implications of such an approach 
are : 

• Principals and supervisor*: must develop their own 
skills of observation, 

• Teachers must help in the determination of the 
type of behavior that is important to observe. 
(Principals a. id teachers must all reach some 
agreement on these behaviors and they must be 
defined in such a way tb*"": cuis agreement can 
be reacx^ 3d . ) 

• Both teachers and principals can anticipate that 
their own views might change. (But a healthy 
optimism should allow a person to perceive this 
as a change for the better — one is not too old 
to learn new things y 

• The post-observation conference between teacher 
and principal or supervisor puts both the teacher 
and the principal in a vulnerable position. The 
teacher is vulnerable because his most important 
skill is being observed; the principal is vulner- 
able because his skill of observation and ability 
to help is being judged as either accurate or 
inaccurate, valuable or not beneficial. Therefore, 
it In extremely important that each approach this 
discussion from a mature view, with goals clearly 
in mind and quite aware of the fact that goals 
might change. 
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Tlie plan fiU' .implementing such an approacli includes ways 
to develop skills In observing, analyzing, interpreting, 
and evaluating information; further, the skills should be 
used in that order . Once the first three are used 
correctly, tlie last one should be much easier to d eve] op. 


Reminders 
Lo InitiaLc 


'File following are reminders which should be heeded to 
initiate such a program: 




e 


Observation aids need to be developed for local 
situations, i . e , , for what is considered to be 
important in given school district. These should 
not be thought of as evaluation sheets which 
criticize harmfully , 




• 


All items of interest may not be observable at 
one time; therefore it may be necessary to observe 
the classroom many times. 




• 


The goal is to describe behavior accurately, not 
to compare behavior against a given reference group 



or in good-bad terms . 

• The ultimate goal is to promote honest interaction 
between the teacher and the principal, between the 
principal, and the central office personnel so that 
all can learn to be more effective. 

Many aspec"“'. of the approach to evaluation described 
are also per Inent to the selection process, since they 
are interrelated in many v/ays , However, another relation- 
ship bears mentioning: research indicates that a teacher 

with an unfavorable attitude benefits less than one who 
views evaluation positively. The implication of this 
research is that one should select teachers who have an 
attitude that will allow them to benefit from evaluation, 

CiCtting To start an integrated personnel program which attempts to 

Started capitalize on the interrelationships among selection, 

evaluation, supervision, and inservice training programs, 
many factors should be considered. To work well, the 
program mu- be suited to the particular organization, 
superviso and principals mus t be trained , and everyone 
must be ..are of the problems which might occur and how 
they can be avoided. Suggestions include; 

• Start on a J.imited scale (perhaps with the admin- 
istrative staff) , 

® Make outside consultants accessible to principals 
and supervisors , 
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• Do not be misled by numerical systems, since 
numbers themselves do not increase validity. 

• Interpret ratings and observations accurately. 

• Observe an adequate number of times, discuss 
observations often. 

• Plan the total system with teachers, special- 
ists, administrators. 





Chapter 2 



PROCEDURRS 



Phases The genera 

improvemen 

research and development information regarding selection 
and evaluation of teachers to be used by practicing 
administrators , teachers, and board of education members 
To accomplish this purpose, it was necessary to acquire 
information, estimate its pertinence, and synthesize 
the information in a written report. Therefore, the 
procedures used included three phases : (a) information 

acquisition, (b) information analysis and interpretation 
and (c) information synthesis and reporting. 

Information Acquisition 

The duration of the project was nine months (December 15 
1969 to September 15, 1970) and the information acquisi- 
tion phase covered the first 7 1/2 months. It consisted 
of the following steps: 

Written Identification and location of written research and 

Information development information: 

0 search of library documents, journals, indexes, 
abstracts, reviews, encyclopedias, and books 

0 computerized search of USOE research through a 
Region IX facility ; screening of printout and 
acquisition of pertinent items 

o computerized search of document s stored in ERIC 
through the Clearinghouse of Educat ional Admin- 
istration at the University of Oregon; screening 
of printout and acquisition of pertinent items. 

0 printout from DATRIX, screening and acquisition 
of pertinent items 

0 correspondence with all ERIC centers and acqui- 
sition of suggested items 

o correspondence with research personnel kno\<rn to 
be involved in related current invest igat ions 



1 purpose of this study was to facilitate the 
t of school personnel practices by providing 



Practices : 
Schools 



Practices : 
Business and 
Government 



Investigation of current practices of public schools : 

• development , testing , and mailing of question- 
naires^^ to 26 of the 40 largest school districts 
in the U, S. and three suburbs of each of these 
large districts, where it was thought most 
innovative practices would be found. The districts 
were chosen in such a manner that all geographic 
regions of the U, S, were represented. A listing 
of the districts is provided in Appendix 2,4. 

• acquisition and analysis of questionnaires* and 
written documents from the 32 districts who 
responded . 

• interviews* with personnel in 30 of the districts 
(representing 13 states) that responded to the 
questionnaire. 

Investigation of current practices of businesses and 
industries , and governmental agencies : 

• development, testing, and mailing of questionnaires 
to 95 medium-t o-large businesses and industries , 

The companies were chosen from a listing in Fortune 
(see Bibliography), representing various types of 
industries ; the initial listing was expanded on 
advice from the Advisory Committee and professors 
of business administration at the University of 
Washington . A listing of the businesses and 
industries is provided in Appendix 2,5, 

• mailing of a similar questionnaire to 19 governmental 
agencies, listed in Appendix 2.6. 

o acquisition and analysis of questionnaires and 

written documents from the 61 companies and govern- 
mental agencies who responded, 

• interviews with personnel in 18 businesses and 5 
governmental agencies in 10 states. 



Information Analysis and Int erpretat ion 

The information analysis and interpretation overlapped 
the information acquisition stage, extending from approx- 
imately January 15 to August 1, 1970, It consisted of the 
following steps : 



^Copies of the questionnaires and structured interview 
forms used are provided in Appendices 2.1, 2.2, 2.3, 



Wr i 1 1 on 
In format ion 



Questionnaire 

Information 



Interview 

Information 



Analysis of written rosearcli and development information: 

• A procedure was devised for classifying and 
assigning items to the researcli staff. 

o As sources were read, ideas were abstracted, 
recorded and filed by author. 



Information was discussed by the research staff 
and a system was devised for organizing and 
relating ideas . 



Analysis of questionnaire informat ion to determine where 
exceptional practices were located and where interviews 
should be conducted. 



o Procedures were designed and checked with the 
Advisory Committee to note exceptional practices 
and make decisions regarding where interviews 
should be conducted . 

a Organizations which deviated significantly from 
the average and the type of activity in which t hey 
deviated were identified from summarized data. 

m The location of exceptional respondents was plotted 
on a map to determine where clusters of exceptional 
organizations existed . 

On the basis of number and type of exceptional practices, 
and location of the organization, decisions were made 
regarding which organizations to visit; an interview 
schedule was then established and appointments made. 

Analysis of interview information and written information 
from organizations. 

• Notes were dictated by the interviewer immediately 
following interviews . 

• Notes were edited by the interviewer (s) . (Note: 
most interviews were conducted by two interviewersc ) 

• Interview notes and written information provided by 
organizations were read and discussed by the research 
staff; unusual procedures and results were identified 
and comparisons among organizations were made. 
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Organizing 
the Ideas 



Preliminary 

Draft 



Final 

Draft 



Summary 



Information Synthesis and Reporting 

This stage of the project overlapped with both of the 
prior stages, extending from approximately April 15 to 
August 31 . It consisted of the following st eps : 

Development of a system for organizing related ideas. 

• Single ideas were identified from initial sources. 

• The relationship of the single ideas to each other 
and to the Initial organizational outline was 
discussed . 

a The organizational outline was revised and the 
single ideas classified according to the new 
outline. 

Production and editing of a preliminary draft of the 
report . 

• Sections of the outline were assigned to members 
of the research staff . 

• Initial drafts were written by individual members, 
discussed and edited by the total staff, and 
rewritten; final editing was done by the principal 
investigator . 

• The preliminary draft was critiqued by the Advisory 
Committee, two graduate classes in educational 
administration at the University of Washington, 

and practitioners in educational administration. 

Production of the final draft of the report . 

• Critiques of the preliminary draft were analyzed by 
the research staff. 

9 Decisions were made for modification of the content, 
organization and format of the report. 

• New assignments for writing were made. 

• Material was rewritten, edited, and reproduced. 

The procedures used in this project allowed considerable 
subjectivity regarding what should be included in the 
final report, i.e., what should be included because it 
is pertinent to the intended audience. However , as 
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information was acquired and analyzed, an effort was 
made to include different points of view, to report 
research results and current practices in an impartial 
manner, and to organize ideas so they would be mean- 
ingful to practitioners. There was also a conscious 
attempt to let the ideas themselves influence how they 
should be organized. Finally, the ideas included in 
this report have been edited and critiqued by many 
people in an attempt to increase the objectivity of 
the final results , 
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Chapter 3 



THE SELECTION OF TEACHERS 



Many oeople consider the teacher selection process as a 
single-stage operation consisting of making a choice 
during an interview. However, the process includes several 
separate but interrelated activities. 

Tasks in the Tasks necessary for completing the selection process 
Selection include : 

Process 

• Determine the total number of teachers needed 
and make a complete position analysis for each 
vacancy . 

• Establish standards for teacher performance appro- 
priate to the situation being considered . 

• Recruit applicants for the positions . 

• Describe the applicants accurately on a variety 

of factors after acquiring various types of infor- 
mation from different sources. 

• Predict behavior of each applicant in the situation 
for which he is applying. 

• Compare predicted behavior of each applicant with 
the desired standards of teacher performance in 
order to make judgments regarding the degree to 
which each standard would be satisfied. 

• Make choices among applicants . 

• Establish controls for the total process by 
analyzing sources of error. 



NOTE 



The seteotion of teachers involves a variety of 
tasksj these tasks may be performed by a single 
person or independently by several people. Wltere 
several people are involved^ eaoh should under- 
stand the total process and his contribution to 
it. (viewpoint: research staff) 
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The tasks in the selection process are designed for persons 
to collect information which may be used to predict on-the- 
job behaviors of applicants and to relate these behaviors to 
the operation of the organization. These tasks are necessary 
to ascertain the relative merit of each applicant for a 
specific assignment, but the tasks themselves do not describe 
the teacher selection process adequately . 



A more adequate description of the selection process includes 
the following items : 

» The person selecting teachers makes an institu- 
tional , rather than an individual , dec is ion . 

• A quota is followed rather precisely. 

9 Information gathered to describe applicants repre- 
sents a variety of dimensions. 

• Usually the decision to select or reject is made 
at any one of various points in the sequence of 
information gathering , i . e . , sequential tests or 
sequential hurdles are used . 

• In all teacher selection decisions, one of the 
acceptable choices is *'rej ect . ” 

• Each teacher applicant is given a single 
assignment . 

This description emphasizes the aspect of deciding which 
applicants are most suitable for a position. However, it 
does not consider what the quota should be or how many (if 
any) of the applicants should be hired at a particular 
time. These problems must be j.ncorporated in the total 
process, but they involve processing of different data. 
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The Development of Criteria for Teacher Selection 



Definition 



How Are 

Personnel 

Needs 

Determined? 



One of the first tasks necessary in implementing the 
teacher selection process is to determine what teacher 
behaviors or results-of-behavior should be used for 
making decisions about which applicant will be hired or 
rejected. However, before these job criteria or standards 
are determined, a school district must identify the 
number of teachers needed to successfully meet its goals. 
Consequently, this section will focus on two major topics 
regarding the teacher selection process : personnel need 

determination and criteria determination . 



Personnel Need Determination 



Need determination is a process through which an organiza- 
tion identifies the personnel, materials, and facilities 
needed to accomplish the goals and purposes of the organ- 
ization. Determination of school personnel needs must 
include consideration of the number as well as the type 
of individuals needed. This is done to match personnel 
with the unique aspects of each position so that goals 
can be accomplished and satisfaction on the part of the 
individual hp'red can be attained. 

Presently , many school districts employ some arbitrary 
pupil-teacher ratio or some extrapolation from prior 
activities to identify the number and type of new teachers 
needed . 

In industry, on the other hand, personnel needs arc deter- 
mined on the basis of a detailed annual plan of operation 
that is updated quarterly or semi-annually. This plan of 
operation generally involves the establishment of broad 
goals at high levels in the organization, which are trans- 
lated into specific objectives at the level which requires 
materials and manpower . * Therefore , needs are determined 
by agreement on objectives and means for obtaining these 
obj ectives ; once agreement is reached , individual units 
can be flexible in accomplishing these goals. 

In school districts where such agreement and flexibility 
exists, individual schools or departments within the 
school may have the opportunity to invest in teacher aides 



^These procedures are often described as "management by 
objectives" and are directly related to Planning, Pro- 
gramming, Budgeting Systems (PPBS) which are becoming 
more widespread in education. 
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or secrctarinl assistance in a way that they consider 
to be most efficient and effective for them. 

SUGHESTION 



Dctemi nati on of school personnel needs 
should be based on an annual plan of operation 
(rather than on some arbitrary pupil-teacher 
ratio or extrapolation from prior activities) ; 
developing such a plan requires: 

p reexamination of goals 

o reexamination of means for accompli. shing 
goals, 

(practice: i n dust ry ) 



In addition to determining personnel requirements accord- 
ing to number and types of positions, many districts now 
take into account the need for teachers who are members 
of certain minority groups, e,g., Chicano, Indian, Black. 
Generally, the basic plan for such needs is made in the 
central office and transmitted to principals, 

A New Needs A new technique which may be applicable to determining 

Determina- teacher personnel needs has been developed by Clement 

tion Approach Bro^ (1968) for determining continuing education needs 
of physicians. The technique is different in that: 

(a) a diagnosis of needs is made prior to "educational 
therapy, " and (b) the continuing medical education program 
is directly related to the needs of patients. More speci- 
fically, the method has three major parts: patient needs, 

medical resources , and physician accomplishments , In 
terms of what can be done for patients, envision two 
partially overlapping circles, one circle designated 
"patient needs" (PN) , the other "medical resources," (MR) 
(See Figure 3.1.) 



PN = patient needs 
MR = medical resources 




Figure 3,1. Patient Needs and Medical 
Resources Overlap . 
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The overlapping area represents realistically those 
patient needs that can be met realistically by the 
current medical resources that are available, or the 
medical resources which are facilitating patient needs. 
Next, envision a third circle, labeled ’’physician 
accomplishments” (PA), which partially overlaps the 
other two circles. (See Figure 3.2.) 



PN = Patient Needs 

KR = Medical. Resources 

PA = Physical Accomplishments 



Figure 3.2. Interaction of Patient Needs, Medical 
Resources, and Physician Accomplishments. 

The overlap of the three circles, or ti;e shaded area in 
Figure 3,2, represents what actually is being done. The 
part of the patient needs-medical resources overlap that 
is not overlapped by physician accomplishment (labeled 
Area 1 in Figure 3.2) indicates the objectives which 
Brora sees for continuing medical education. That is, 
rather than spending time dealing with continuing medi- 
cal education programs which stand little chance of 
helping those patients who, for example, have non- 
preventable disabling diseases. Brown’s needs-det ermina- 
tion model points out the necessity to identify those 
disabling diseases that can be prevented but which 
physicians are not handling adequately. In short, the 
technique is a rational approach which allows for setting 
continuing education priorities. 

Adaptation In adapting this model to determining school personnel 

t-o EducatioK needs, envision student needs in place of patient needs, 
educational resources in place of medical resources, and 
school or staff accomplishment in place of physician 
accomplishment. (See Figure 3,3.) 
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ER = Educational Resources 



SN = Student Needs 



SA = School Accomplishments 



Figure 3.3. Interaction of Student Needs, Educational 
Resources, and School Accomplishments. 

The overlap of the three circles (i.e., the shaded area 
in Figure 3.3) indicates what the school or staff is 
currently doing to meet students^ needs. The part of 
the student needs — educational resources overlap that is 
not overlapped by school or staff accomplishment (labeled 
Area 1) suggests those student needs for v/hich resources 
are available and which realistically can be accomplished 
by the school or staff if proper attention is given. 

To implement this technique the following procedures 
should be followed:* 

• Have teachers and principals identify the needs 
of their students. 

o Have teachers and principals identify those needs 
that education has the resources and ability to 
meet . 

o Have teachers and principals identify those student 
needs that are being satisfied by the personnel 
currently available. 

o Based on the residual, identify areas in which 
current personnel can be given additional train- 
ing, and establish a search for new personnel to 
fill voids current staff cannot fill. 

This last step is designed to help fill the vacant space 
of the student needs-educational resources overlap. 



*Tliese procedures are adapted from a paper delivered by 
Brown at the Conference Workshop on Regional Medical 
Programs, Washington, D. C., 1968. 
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In short, this procedure provides the personnel worker 
with some assistance to rationally determine personnel, 
as well as materials, facilities, and inservice training 
needs. It also helps identify those student needs which 
the school should be meeting given the current educational 
resources, but which are not being met because of lack of 
application of those resources by the school or staff, 

SUGGESTION 



Determiyiation of school personnel needs may he 
based on an analysis of students^ needsy edu- 
cational resouroesy and school or staff 
accomplishments n ( viewpoint: Browny 1968) 



Overhiring 

\^at happens when overhiring has occurred? How does this 
situation affect the selection process? \^Jhen overhiring 
has occurred, industry has approached the problem differ- 
ently from most school districts. Since industrial needs 
fluctuate rapidly at times, some industries have attempted 
to control the need for hiring by initiating training 
sessions for employees during slack periods rather than 
reducing the work force and then spending money for re- 
acquisition. 

SUGGESTION 



When overhiring occurs y school districts should 
consider using available personnel in alterna- 
tive ways (rather than simply modifying the 
pupil-teacher ratio) in order to increase the 
long-range effectiveness of the system^ 

( prac tice : indus try ) 



At least two possibilities are reasonable in most situa- 
tions of this type: 

o Special training sessions can be established for 
people who are having difficulty or who need to 
acquire special skills, 

• Certain persons on the staff can be released to 
accomplish special needs such as research. 
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Although decision problems would be created in 
determining who should be involved in the 
special activities, this procedure is poten- 
tially more beneficial than simply modifying 
the up il- teacher ratio . 



NOTE 



Accurate estimates of personnel needs some- 
times are influenced by the consequences of 
previous misestimates, (practice: schools) 



Ifliere principals are responsible for estimating needs for 
personnel in a building, and when they are not likely to 
he required to transfer teachers to another building if 
an overestimate occurs, they tend to be liberal in their 
estimates of enrollment (and consequent teacher needs) • 
Conversely, conservative estimates tend to occur when 
transfers are required. 



Criteria Determination 



Once the decision has been made to hire new personnel, it 
is necessary to determine criteria to be used in selecting 
new teachers. To accomplish this task, the following must 
be done: 

• The nature of the criteria to be used must be 
understood. 

• The important aspects of the position (s) to be 
filled must be analyzed. 

• The best means for deciding on which criteria 
will be used for specific teaching positions 
must be determined . 

Doing these three things effectively increases the proba- 
bility of choosing the best teacher for each job. 

Nature of criteria . \^^nat is a criterion or what are 
criteria? A criterion is defined as a standard, rule, or 
principle, which is used to judge the value or utility of 
something. That is, it serves as a standard against 
which comparisons may be made, e.g., a criterion of a 
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good major league baseball pitcher may be that he wins 
at least twenty games a season. 



In teacher selection, a criterion is convSidered to be 
those behaviors or result s-of-behavlor which the 
personnel worker is trying to predict. The personnel 
worker attempts to determine certain attributes or 
performances of prospective teacher candidates which will 
assist him in selecting those individuals who can success- 
fully perform on-the-job tasks. Consequently, in the 
selection process, the personnel man initially specifies 
standards of success! ul teaching for a given situation ; 
he then identifies, through study and experience, those 
variables which will help him predict which candidates 
will meet the criteria of a "good” teacher for this 
specific situation . 

Criterion Historically, the teacher selection process has been 

Problems geared to predict only a single criterion. Inherent in 

such an approach has been the assumption that there 
exists a single ultimate standard expressed by some 
general overall factor which represents what a "good" 
teacher is . Although the use of an ultimate criterion 
would be advantageous, the fact is that such an outlook 
is a distortion of reality since the nature of the task 
to be performed has many complex facets and is constantly 
changing because of new inputs. 

The fallacy of the single criterion lies in 
its assumption that everything that is to 
be predicted is related to everything else 
to be predicted — that there is a general 
factor in all criteria accounting for 
virtually all of the important variance 
in behavior at work and its various con- 
sequences of value . (Guion , 1961:145) 

It has been known for some time that criteria change 
from time to time and from one situation to another, and 
that there are many aspects contributing to job success 
(Guion, 1961; England and Patterson, 1960; Ghiselli, 

1956; and others). For example. Forehand and Gilmer 
(1964) found that criteria of job performance change, not 
only as a result of the growth and development of a 
particular job position, but also because of the vigorous 
functioning of the organization over a period of time. 

A case in point is the research by Bass (1962), which 
showed that after three years, meritorious performance of 
good salesmen was morf contingent on esteem and popularity 
than on ability. An ^^xample of the effect of organizational 
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Tactiors on teacher beliavior is tlie study conducted by 
Wilk and Edson (1963) in wliich tliey looked at student 
teacher responses to multi-*sized groups. They found 
tliat classroom placement variables had a significant 
influence on student teacher performance , 

Recognition of tlie complex and changing nature of 
criteria concerning teacher selection has been evident 
for some t ime , Exemplars of this recognition are quite 

evident in the literature regarding teacher effective- 
ness, e.g,, see N, L. Gage’s chapter, ’’Paradigms for 
Research on Teaching," in his Handbook of Research on 
Teaching (1963) or the Flanders article, "Teacher Effec- 
tiveness," in the Encyclopedia of Educational Research 
(1969). 

NOTE 



In developing criteria for teacher selection^ 
consideration m^st be given to the complex 
interaction of teacher behavior^ learner beha- 
vior^ a?id. enviroynmental factors in the teaching- 
learnim process. In addition ^ it must be 
recognized that with the passage of time these 
individual and situational variables change. 



Understanding that the nature of criteria for teacher 
selection are complex and dynamic, how does one go about 
identifying and deciding what the actual criteria should 
be'? It is the purpose of the next two sections to deal 
with this question. 



Job dimension specification . The first step in identi- 
fying criteria for teacher selection is development of 
a position analysis from which to infer the criteria 
important for job proficiency . The purpose of the posi- 
tion analysis is to gather and analyze data about the 
nature and conditions of a position so the personnel 
worker can: 

o identify what is important for selection and 
evaluat ion purpo ses 

• determine how to measure what is important 

# communicate to the prospective new teacher 
the major and minor aspects of the position 
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Scope of 
Position 
Analysis 



0 provide guidance for setting up inservice training 
programs 

• infer what information might be collected that 
would help predict the success of the applicant 
on the job. 

The scope of position analysis, in industry , generally 
includes both an analysis of the ( e.g. , the job 

title, the procedures of work, the physical ccndltlons 
of work, the relation of the job to other jobs, and the 
conditions of employment) and an analysis of the employee 
(e,g., physical characteristics, psychological character- 
istics, and demographic data) . 

The scope of position analysis, in education , is somewhat 
similar to industry and government; however, as Gilbert, 
et al . (1966) indicate, following their survey of 320 
large school districts in the United States, only five 
percent used a specific job description form, and there 
was a trend which suggests that the size of the school 
district increased, the specificity of information in the 
j ob descriptions decreased , 

All too often, position analyses are superficial and 
global. A common tendency is to stereotype desirable 
qualities in a teacher; however, lists of general qualities 
are of little value to criterion development. What is 
needed is an accurate description of the job, and, as 
Thorndike (1949) suggests, ”..«an examination of worker 
characteristics in relation to the job. ” 

It is important that position analyses be made and updated 
frequently for each teaching position (Ellsbree, 1959; 
Mahoney^ et al . , 1963). The major reason for this results 
from the nature of criteria, which is affected by the 
unique aspects of each t caching situation and the complex 
and dynamic relationship of the existing environmental 
factors to the behavior of teachers. 

In conducting an analysis of a position, the basic question 
that is trying to be answered is, ”lVhat must the indivi- 
dual do and what characteristics must he possess to be a 
success?” Basically, this is the same question instruc- 
tional designers ask in the development of curriculum and 
course objectives, i.e., ”What must the learner do to 
successfully attain the instructional obj ectives , and 
what prerequisite knowledges or skills must he possess?” 
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whether tlic object of focus is selection or instruction, 
the answer to the question, *hVhat is important,.,*’ 
provides tlie essential criteria for success. 

Palmer (1970), in studying the effect of using a completed 
position analysis outline (PAO) on selection decisions 
in a simulated teacher selection situation concluded that 
the PAO had an effect on the behavior predicted and 
ranking of certain applicants. This result indicates the 
potential for using a PAO in the teacher selection 
process. The fact that the completed PAO provided sub- 
jects with a more explicit criteria upon which to base 
selection decisions was consistent with the notion that 
different applicants may be viewed differently as **a 
result of explicit criteria** (Palmer: 1970, A7). 

The following position analysis outline is the one developed 
by Palmer in his research: 

I. General Nature of the Position 

A. Broad goals of the system 

B. Organizational structure of the system 

C. General expectations of the teacher 
In the classroom 

In relationships with members of the 
organization 

3. In relationships with external groups 
or individuals 

II. Static and Dynamic Features of the Position 

A. What is likely to be more important at 
the beginning of the assignment? 

B. What is likely to change? 

C. What is likely to influence this change? 

D. How will this position be affected by 
other people? 

How accepting is the community? 



;ii. 



IV, 



1 . 

7 



E. 

F. 



IVha 


t is 


the nature of 
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Behaviors 
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uired 
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B. 

Teacher Characteristics Sought 
A. Aptitudes 
R. Skills 

C. Social requirements 

D. Interests 

E. Physical 
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A 



SUGGESTION 



Final 

Criteria 

Decisions 



Because of the complex and dynamic iviture of 
each teaching position^ an accurate and detailed 
position analysis^ developed for each position^ 
aids in making decisions in the selection of 
teachers. (research: Palmer^ 1970) 



Deciding on criteria * Once the position analysis has been 
made, the next step in criteria determination is to use 
the information to help decide on what the criteria or 
standards will be to indicate job success. 

What should be considered in choosing criteria? It is 
recognized that criteria used in teacher selection 
include both teacher behaviors and/or result-of-t eacher 
behavior. \^en one is identifying the criteria to be 
employed for teacher selection, the following ideas 
should be considered : 

• A limited number of behaviors should be identified: 
this increases the reliability of measurement by 
staying within the capacity of observers. 

• Behaviors should be reliably measurable, opera- 
tionally defined, and focused on specifics. 

• Behaviors should differentiate among individuals 
and ought not to overlap. 

Q Behaviors should be related to the purpo ses of the 
situation in which they are expected to be exhibited. 
Behaviors that are considered important must be 
established in each community — in light of an 
accepted value system. 

• Behaviors should include verbal interactions as 
well as non-verbal communications, in the class- 
room as well as out-of-classroom action. The 
manner in which activities are organized, choice 
and content, types and sequences of activities 
and strategies used in teaching are all means of 
producing certain effects on learners , and are 
TvCasonable items to include for observation . 
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CAUTION 



BecauC'C teacher ors to be predicted are 

complex y dynami.c^ and specific to a situation, 
long^lasting predictions are not likely to 
occur. BecaxA.se of this, the usefulness of 
certain applicant characteristics to predict 
on- the- job success should he reviewed, 
frequently . (viewpoint: Guion, 1981; 

and others) 



Conditions change under which individuals teach, and 
this affects the behavior that they exhibit and the 
outcomes they produce. For example , the type of super- 
vision in a given building might change drastically with 
a change of principals ; consequently, more or less 
assistance with certain activities might be expected. 

Also , behaviors considered inport ant might change as 
the student population changes , The changes in these 
two factors, conditions of work and nature of the student 
population, are instances of the changing or dynamic 
nature of the teacher behaviors being predicted and 
account for some of the difficulty of obtaining long- 
lasting predictions. Therefore, it is necessary to 
reexamine the established criteria for each teaching 
position periodically . 

Criteria How should criteria be defined? To insure the quality of 

Definitions selection decisions, the criteria must be defined by 
specifying clearly the behaviors or outcomes to be 
measured and the level of performance necessary for 
success, 

NOTE 



Clearly defi^ied criteria not only serve as 
standards against which comparisons are made, 
but also guard against misinterpretation, 
(viewpoint: research staff) 



'A more technical discussion of this criterion validity 
problem is presented in Appendix 3.1. 
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Who Decides? 



Who decides on the criteria to be employed and the 
assigning of the weight to each of the criteria? Gen- 
erally, the final decision is based on a value judgment 
of some individual or group. As a result, the worth 
of the value judgment, and the selected criterion, is 
dependent on the expertise of the decision maker (s) and 
the information available for determining the criteria. 

NOTE 



Decisions on criteria are improved when 
based on judgment pooled following empiric 
cal validation. (research: DeNelski/j 1969) 



If an individual’s intuitive judgment, expertise notwith- 
standing, is the only recourse for determining criteria, 
the decisions being made are ’’built on the weakest of 
foundations ” (Guion, 1961). However, there are indica- 
tions that if individuals pool their judgments, the 
selection of the criteria will be improved. DeNelsky 
(1969) found that the pooled judgment of several judges, 
using a modified version of Stephenson’s 0-sort technique, 
yielded greater predictive accuracy that the judgments of 
individual psychologists. 

Various statistical weighting techniques are suggested by 
Ghiselli and Brown (1955) as one means to solve the 
problem of experts subjectively choosing and weighting 
criteria. However, as Albright;^ al. (1963:37) point 
out, ’’until a comprehensive theory of job performance is 
formulated [the pooled judgment of experts] is probably 
as good a method for the practitioner as any.” 

Involvement of teachers in the development of criteria, 
and specifically the indication of behaviors desirable 
for a given position, generally occurs in collaboration 
with principals. It appears that principals often have 
the responsibility of precisely defining the job descrip- 
tion and they share this responsibility with teachers. 

^ CONCLUSION 



Involving teachers in the development of 
criteria may serve two purposes: the posi- 

tion may be defined more accurately ^ a'ld 
the morale of the professional staff may 
be increased, (practice : schools) 
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In addition to principals and teachers, tlie following 
personnel are sometimes involved in development o f 
criteria : vice-principal ; department chairman ; central 

office personnel, including superintendents or tlieir 
representatives; personnel directors; and curriculum 
supervisors . Wio makes the final decision regarding 
criteria determinat ion indicates where the district 
has placed responsibility for development of pliilosophies 
and means of operation, 

KHien the central office controls the final determination 
of criteria, uniform selection criteria are likely; hut 
where building principals make decisions regarding 
criteria, there is likely to he considerable diversity in 
characteristics being sought . 

CONCLUSION 



Where the decision is made regarding deter- 
mination of criteria is likely to influence 
the diversity of criteria bei^^g sought, 
(position: research staff) 



Summary 

Regarding 

Criteria 



In summary, decisions regarding criteria selection should 
be made with the following ideas in mind: 

• The behaviors desired or the result of the beliaviors 
(i.e,, the criteria) should be capable of being 
measured reliably , 

• The criteria should be relevant to the specific 
situation, that is, they should have acceptable 
validity , 

• Criteria should be updated frequently because each 
local situation is complex and changes rapidly, 

• Criteria should be acceptable to teachers as well 
as to administrators and other members of the 
community , 



Rccr^'it-ment 



Each school district, once having established general 
needs and developed more specific job descriptions within 
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the framework of those needs, cannot actually bep Ln the 
selection process without a pool of candidates from 
which to choose. Because selection implies choice, 
recruitment or the search for enough capable personnel 
is crucial to the selection task. 

In addition to the development of an applicant pool , 
recruitment is crucial to selection because it is the 
basis of the quality of candidates as w"-:!! as the 
quantity . 



Reovuitment must attract both a high quality 
and an adequate quantity of applicants if it 
is to enhance the selection process* (view- 
point: research staff) 



Generally, the intensity of recruitment is related to the 
availability of potential candidates from tie teacher 
market. 



When a surplus of teachers exists^ school 
systems use rigorous selection procedures; 
however^ when there is a shortage of candi- 
dates^ school systems must develop and 
maintain an effective recruitment program 
as a prerequisite to selection* (practice: 
schools) 



It is axiomatic that no selection program can be effective 
unless the number of candidates is substantially greater 
than the number of positions (Fear; 1958), i.e., a system- 
atic selection program cannot compensate for an inadequate 
number of candidates. A recruitment propram must provide 
an applicant pool that is adequate in nur'her and diverse 
in characteristics. \Vhen the applicant pool is too small, 
the problems are obvious; and most administrators try to 
irske positions more attractive or to contact more potential 
applicants. \^en the applicant pool is at least adequate 
or very large, there is a tendency to be complacent. Often 
a careful examination of the recruitment process does not 
occur. 



NOTE 



NOTE 



91 




CAUTION 



Although an appHoant pool may he large ^ 
e'oidenoe should continue to be oolleoted 
to determine whether modified recruitment 
procedures might produce more diverse and, 
more highly qualified applicants. (prac-> 
tice: industry) 



The large applicant pool may not always contain candi- 
dates to fit special needs nor does it guarantee enough 
high quality candidates. 

Because recruitment is continous, fast moving and competi- 
tive , recruitment programs t end to require considerably 
more money and a disproportionate amount of administra- 
tive time (Maloney: 1961), partly due to the fact that 

some administrators like to maintain personal control of 
this phase of selection. However , the absolute cost of 
recruitment should not be considered separate from the 
results , and there is a continuous need for school dis- 
tricts to monitor recruitment procedures, to measure 
':heir effectiveness, and to make appropriate modifications. 

NOTE 



The cost of hiring the wrong candidate 
can be higher in terms of supplementarv 
training, wasted, salary, adverse public 
relations , and. lost productivity than 
the cost of more ex.tensive recinAitment 
would have been. (practice: industry) 



Many businesses will expend vast amounts of money and 
effort each year on personnel recruitment in order to 
increase productivity and reduce the cost of finding and 
orienting replacements (Hinrich: 1960). School districts 

may wish to adopt this business practice. 
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SUGGESTION 



School districts should analyze their recruit- 
ment programs in terms of cost versus results. 
Such an analysis may increase initial costs ^ 
but more effective recruitment programs should 
produce better lonO'-range results, (practice: 
industry) 



A recent study of recruitment practices in some randomly 
selected districts in a midwestern state found that the 
average cost per teacher hired amounted to $146 (Fitz~ 
gerald: 1970), A remarkable and especially interesting 

comparison occurs when this is compared to the financial 
effort put forth in busine s and industry, $1822 per 
professional hired • 

Fitzgerald also found i lack of written policy and budget- 
ing for recruitment in the districts studied. The pro- 
ject found that often the recruitment program is looked 
upon as only a scheduling of interviews on a travel 
itinerary (with occasional advertising). It would seem 
that if recruitment is important enough for the kind of 
expenditures noted by business and industry; that if 
schools do not look upon recruitment as important enough 
for policy and budgeting ; and that if school officials 
are not seeing recruitment as part of the larger selec- 
tion process, school officials should begin to seriously 
examine their recruitment efforts and begin local research. 

Recruitment serves more than the immediate function of 
strengthening an applicant pool. Although forecasts are 
not clear as to the duration of any current teacher sur- 
plus, there seems to be a trend for teacher-training 
institutions to be more selective in accepting trainees. 

Because it may become necessary for school districts to 
attract those fewer but better trained teachers being 
produced, recruitment will remain essential to the selec- 
tion process. If a large surplus remains, recruitment 
will still serve its function of attracting high quality 
candidates . 
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SUCnESTION 



Diverse recruiting practices should he 
developed for both long-term and short- 
term needs. (practice: industry^ schools) 



To accomplish long-term and short-term needs in recruit- 
ing personnel, the following suggested procedures are 
being used by industries, government agencies, and 
school districts. 

• Develop a close relationship with colleges and 
universities, in order to let colleges know 
what positions are available and what types 

of people are needed, and to influence the 
curriculum being offered to future employees. 

The procedure for developing close relat ion- 
ships varies, but includes ; 

a . sending the same recruiters to the same 
campuses annually so they become acquainted 
with campus personnel ; 

b. sending recent graduates back to the alma 
mater to talk to seniors and tell campus 
personnel how well they like their position, 
and 

c. providing each campus with a cumulative 
listing of graduates who have been hired 
by the organizat ion . 

• Sponsor a two-day tour and visit of the district 
for placement directors. 

9 Use minority employees to visit colleges to help 
identify prospective candidates in minority groups. 

t Participate in "career day" programs on campuses. 

• Provide supervisors , t eacher s , or principals to 
talk to classes or groups of students on campuses. 

• Send curriculum materials to colleges and univer- 
sities for use in curriculum classes. (In industry, 
this takes the form of sending products to campus 
for examination. ) 
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• Encourage employees to ask friends and acquaintances 
to apply for employment; in some cases, incentive 
bonuses are given to employees if a referral is 
employed. 

• Establish a temporary recruitment center in a 
given geographic center. Advertise in advance 

to attract inquiries, and send teams to interview 
those who make requests. This is a procedure 
used to attract experienced teachers. 

• Use a 10-15 minute slide-tape presentation before 
the interview to acquaint the applicant with the 
district and give him a more realistic view of the 
community (or prepare a brochure on the district, 
including not only salary schedules and promotion 
policies, composition of student population, etc., 
but also Information on living expenses.) 

• Advertise in Afro-American newspapers to attract 
minority group teachers . 

The lack of research on recruitment practices indicates 
that effects of particular strategies are taken for 
granted. 

SUGGESTION 



Local school districts should invest in 
research to identify effective recruitment 
procedures. (practice: industry) 



Collection of Data on Applicants 



The selection process and its sequence are based on the 
collection of data on applicants. Without a systematic 
means for collecting and analyzing information, the 
differences between candidates are less apparent and 
precise. Without data the selection decision becomes one 
of chance. 

The selection decision has multiple sources for the infor- 
mation needed. The following list comprises the main 
sources of candidate data: 
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0 letters of application and inquiry , and resume 

0 the application form and personal history 
questionnaire 

0 recommendations 

0 college placement and commercial agency materials 
0 c er t if icat es and licenses 
0 interview (s) 

0 paper and pencil tests 
0 actual performance assessment 



/ 

Letters of Application and Inquiry, and the Resume 

Although little research information has been found on 
these sources of information in relation to selection 
decisions , they are usually unsolicited and the first 
contact (other than possible recruitment interviews) 
frr . a potential applicant. However, the information 
they supply may be differentiating to some degree. For 
instance , it could be hypothesized that the spelling 
and clarity of these initial sources of information may 
indicate some candidate quality (or lack of it). 

CAUTION 



Any decisions made at this point should only 
be made on the basis of rep eotion of the 
obviously incompetent. (practice: schools, 

industry) 



NOTE 



Businesses ha.ve indicated that there may be 
som,e value in evaluating the applicant's 
letter in terms of the information he is 
seeking and what information the applicant 
has about the employing organization which 
may liave prompted the inquiry. (practice: 
industry) 
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The resume is included in this first group of infor- 
mation sources because it is usually unsolicited and 
accompanies the letter of inquiry. 

CAUTION 



The resume may not always he a reliable 
sourae of information I'ecause of the 
variety of acceptable formats and because 
its purpose is to make the potential can- 
didate look attractive. The resumi may 
have been written by a professional agency 
rather than the candidate. (practice: 
industry) 



The lack of specific information on these sources of 
data should also indicate the need for school districts 
to conduct research. 

SUGGESTION 



School districts should engage in research 
to justifv the continued use of letters of 
application^ inquiries^ and resumes as infor^ 
mation for making selection decisions. (view- 
point: research staff) 



Application Form and Personal History Information 

One of the common personnel practices in industry on 
which considerable research has been conducted is use 
)f the application form, or its companion, the personal 
history questionnaire (Dunnette, 1959 ; Heilig, 1963; 
Edgerton, e^ a^. ,1957) 

CONCLUSION 



School districts do not capitalist n the 
potentially beneficial predictive informa- 
tion contained in the application form and 
personal history questionnaire. (practice 
industry } 




The use of the application form as a source of informa- 
tion to predict a candidate’s success is based on studies 
which usually use current employees as subjects. Gen- 
erally these studies correlate information from the 
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application blank (often called demographic data: sucli 
as age, marital status, number of dependents, years of 
experience, etc,) with some measure of success (super- 
visor ratings, salary, etc,). 

These correlations become the basis for predictive 
validity estimates , Tliose items on the blank which 
correlated significantly with '^success" items are tlien 
used to predict the success of new applicants. 

Ileilig (1963) completed a correlation study between 
application form data and teach ag success. The criterion 
was the department chairman’s judgment of ’’success.” He 
found that the age when hired, teaching professional 
society membership , publicat ion , and former military 
rank were the most frequently uccurring predictors of 
success in individual departments. Specific character- 
istics for individual departments or divisions of the 
employment group did vary and "characteristics accepted 
as predictors of teaching effectiveness did not apply in 
some departments , ” 

SUGGESTION 



Since application form data do not predict 
success equally well for all positions^ 
local school districts should, develop and 
validate their own fonns. (vie^opoint : 
research staff) 



In contrast to the weighted application form, the "personal 
history" correlates various background topics with cri- 
t er i which deal with human relations effectiveness and 

creativity (Edgerton, Feinberg and Thomson: 1957). The 

predictions made in these studies are concurrent validity 
estimates; they can cover such items as preferences, 
attitudes and interpretation of experience rather than 
being limited to factual items so characteristic of the 
studies of weighted forms. 

Typical of some of the studies conducted in industry is 
one done by Smit^ ^ (L963 ) who found that the per- 

sonal history technique was valuable and that it yielded 
noncurrent validity estima tes of ,61 and .52 for overall 
performance ratings . 



Recommendations 
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Most studies indicate that letters of recommendation are 
either v/orthless generalizations or have not kept up with 
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the changes in skill of the applicant. Stone and Kendall 
suggest that a request for references should cover speci- 
fic questions such as dates of employment, salary, 
whether former employer would rehire, basis of judgment, 
etc • , and should seek an appraisal of performance . Most 
if not all letters of recommendation carried by the appli- 
cant are worthless in obtaining an accurate and unbiased 
appraisal of his personal character or worth to an 
employer (Stone and Kendall: 1956). 

With regard to written materials, the U. S. Civil Service 
Commission (1961 ) cautions the employer to be sure that: 

0 elements in his inquiry form correspond to the 
more important qualifications of the posit irn; 

• he has obtained information from as many persons 
as possible who have observed the applicant in a 
variety of situations during, as a minimum, the 
past five years; 

a he includes a letter, signed by the responsible 
executive , with the form, emphasizing the impor- 
tance of the evaluation process, presenting a 
description of the job to be filled , asking for 
full cooperation, and stressing that the form is 
confidential and will be used only to evaluate 
the candidate; and 

• he gives greater credence to forms where the 
narrative indicates that the rater has full 
information, has seen the applicant in a variety 
of situations , and has used high standards in 
his evaluat ion . 

Studies about letters of recommendation do indicate that 
they contain some worth if they contain discriminating 
adj ectives. 

NOTE 



Letters of recormendation from previous 
employers whiah describe specific mental 
ability and precise job success are much 
more discrimznating than those which 
merely damn with faint praise. (research 
Peres and Garcia: 1962) 
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Peres and Garcia (1962) found tlial: words such as 
'^cooperation” and "urbanity” were poor discriminators 
in terms of job success. In their study of G25 reference 
letters , factor analysis identified five factors : 

(a) cooperation-consideration, (b) mental ag;llity, 

(c) urbanity, (d) vigor, and (e ) dependability-reliability. 
In another study, Bozarth (1956) found that descriptive 
comments in the reference letter had little relat ionstiip 
with scoves on the Miller Analogies Test and the Vlatson- 
Glaser test of critical thinking, 

^ NOTE 



The problems yKth using letters of recom- 
mendation in the selection process seem 
to be: 

# Validity varies greatly according 
to tha quality of the letters. 

0 Most letters do not discriminate 
sufficiently co^ioyg applicants. 

{practice: schools^ industry) 



This suggests that organizations should begin to train 
those responsible for the reading and writing of letters 
of recommendation and begin some local validation of 
those letters by matching them against the criteria of 
success previously mentioned. In this way, both consis- 
tency of quality in and power of prediction from reference 
letters should be increased , 



Placement Agency Data 

For predictive studies not unlike those developed with 
application forms, information from the candidate's plac(^- 
ment agency materials may be wortli exploration. Although 
information from the candidate’s placement agency may 
include information redundant to the application forms , 
it can be used as a check against the application data. 
Placement agencies also furnish transcripts, student teach- 
ing evaluations, and references and recommendations from 
college personnel and/or former employers. 

Research evidence regarding the valid use of agency infor- 
mation is seriously lacking, and because of the diverse 
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nature of the agencies and the information they supply, 
the consistent predictive power of that information is 
questionable, 

^ SUGGESTION 



School districts should seek a common under- 
standing with placement agencies regarding 
the need for development of consistently 
reliable information on candidates. (vieio- 
point: research staff) 



Certification 

It is usually the candidate’s responsibility to present 
evidence that state and local legal requirements have 
been met and certification or licensing has taken place. 
Teaching certificates are nearly always prerequisites to 
being hired; therefore they serve to screen out those 
applicants who do not legally qualify for consideration. 

Intervi ews 

Purposes of No part of the selection process is more widely used than 

Interview the personal interview; most people feel that the inter- 

view contributes something to the selection process that 
cannot be gained in any other way (Fear, 1943). Perhaps 
this feeling is the result of the personal contact that 
provides a two-way exchange of information or the oppor- 
tunity to give information as well as get it (Moyer, 
1948; Horst, 1962; Wagner, 1949). Certainly the person 
interviewed is left with an impression, correct or not, of 
the entire organization (Moyer, 1948; Hansen, I960), 

NOTE 



Interviews are designed to acquire information 
about applicants at%d to provide information to 
them; the type of information heir sought and 
provided decides the structure ana content for 
the interview. (practice: schools^ industry) 



For example, v/here the purpose of the interview is to 
determine if the individual's aspirations are aligned x^7ith 
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a vacancy in the districti certain questions and areas 
of discussion are suggested. Also, the interviewer can 
ascertain that the applicant fully understands the work 
involved, thereby decreasing later turnover from job 
dissatisfaction. The amount of time spent in presenting 
information about tlie school district depends on the 
scarcity of applicants and the nature of inforr.ation 
provided in other formats. Some distri^s and businesses 
use written brochures or slide-tape presentatiot's to 
supplement the interview. 



Interview Wien designing the total interview procedure, many differ- 

Design ent factors should be considered, among them the following 

• number of intervie wers to be simultaneously involved 
in an intervi.ew 



• which people to involve as Interviewers 



• 


number 


of 


int 


er views to be conducted per 


candidate 


a 


number 


of 


int 


erviewees to be observed at 


one time 


0 


nature 


of 


the 


interview 





• relation of written information available about 
the candidate to the interview 

NOTE 



The moy.t outstandhig oharaotevistiQ about 
interview practices throughout the country 
is their variety. (practice: schools^ 

Indus t2vy) 



The structure of interviews varies from virtually no 
structure at all, with great flexibility available to 
the interviewer, to highly structured interviews where 
questions to be asked are decided beforehand. One per- 
sonnel director reported that he allows each Interviewer 
to design the interview to fit his own personality. 

Several industries make use of on-site visits which allow 
the applicant, as well as his potential supervisor, to 
become well acquainted with the future working conditions. 
In some places semi-patterned interviews are conducted to 
minimize overlap of questions by different interviewers. 

In that way the applicant is not compelled to answer the 
same set of questions in a series of interviews. 
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Interview 

Sequence 



Interviewer 

Training 



Some idea of the variety 5n the number of persons who may 
Interview a single applicant , as well as the different 
patterns of interview sequence, is presented in Figure 
3.4. The first box in each sequence indicates thu source 
of initial contact. In some cases this may be simj^ly an 
unsolicited letter, it may be a letter solicited by pub- 
licity sent to Li)e college, or it may be a regularly 
scheduled interview on campus. 

Teachers sometimes help to determine how the applicant 
will work in a team situation. Also, they may help 
"sell” an applicant on a ^ osition or provide information 
to assist him in making a decision. 

Usually, at least one interview is arranged v/ith a person 
high in the organization, another with the principal cr 
the academic supervisor, and sometimes a neer is utilized 
in order to give the applicant a complete picture of the 
organization. Sometimes princrpals or consultants are 
asked to make composite ratings on applicants subsequent 
to ratings on specific qualities. \\Tiere this happens, 
the personnel department usually analyzes the data and 
makes a final decision. hHicn the final decision is not 
made by the personnel department, the principal is often 
asked to indicate a preference among applicants. 

Industry makes use of similar sequences; the head of the 
personnel department makes similar decisions to those of 
the superintendent of schools, and the department super- 
visor plays an analogous role to that of the principal. 

In addition, many industries are concerned that applicants 
have done some "homework" on the industry, demonstrating 
their interest by acquiring considerable information 
about the company before making application for v/ork 
there . 

Another source of variety in interview practices is the 
amount of training givc.n to inter\iewers . One school 
district reported a full two-day v/orkshoip is devoted to 
traj ning. More generally, scnools and industries tend to 
devote an hour or two at the most; some were as casual as 
"an informal discussion in the car on the way tc the 
college to interview." The use of training manuals for 
interviewres was found in a few cases. Occasionally, if 
untrained interviewers were used, a trained or experienced 
interviewer accompanie'.d them. 
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Sequencer A 



Sequence B 



Sequence C 



Sequence I) 




Figure 3,4. Typical Sequences o2 Interviews 
in the Selection Process. 



^usually only in small school districts 
104 
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Use of 



NOTE 



Single 

Criteria 



Group 

Interviews 



Many sohool districts and industries still 
erroneously use the interview as a way to 
uncover the single criterion which they 
believe predicts teacher or employee 
effectiveness. (practice: schools^ 

industry) 



For example, seme districts expressed the desire for 
teachers who cared more about children than subject 
matter, or who were enthusiastic (believing that enthusi- 
asm overcomes most difficulties) or who were of a specific 
race. There seemed to be, in many instances, little 
recognition of the need for multiple predictors, as 
well as predictors specific to each situation and group 
of students. 

Industries tend to make more use of group interviews (e.g., 
where more than one interviewee is observed as a discussion 
is held on a given topic) than do school districts, but 
neither schools nor industries use them extensively. The 
"serial*^ interviews shown on Figure 3.4 are quite commonly 
used in the schools. 

As an example of a group interview, the Leaderless Group 
Discussion technique is used to select persons exhibiting 
leadership ability. Six to eight examinees are asked to 
discuss an assigned topic for a specific length of time. 
Raters , who do not take part in the discussion, observe 
their performance. 

Bellows and Estep (1956); Bass (1954); and Willing (1962) 
cite several advantages of the group interview: 

• All applicants are seen simultaneous!’ '^y all of 
the people responsible for the selection decision. 

• Multiple observations of the same behavior tend to 
increase reliability. 

• Less time is used by each observer. 

• People who find interviewing difficult because of 
poor face-to-face human relations skills may 
able to observe and rate in this manner. 
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St rue Lured 
Versus 

IJns t rue Lured 
Lntervi cv/s 



Some invGsLiftators believe that nondirective nr unstruc- 
tured interviews result in better evidence that more 
direct questioning (Richardson , et al . , 1965; Clirudcn and 
Sherman, 1961) . VJlien the applicant is allowed to ta.l k 
about whatever seems important to him, it is believed be 
will provide the essential information, even though each 
applicant provides different information (Moyer, 1948). 



Other investigators state that only through the structured 
interview, which facilitates collection of comparabl.e 
information among applicants, can reliable and objective 
decisions be made (Ulrich and Trumbo, 1965; England and 
Patterson, 1960; Crissy, 1952; Maccobby and Maccobby, 
1954). In practice, most interviewers use varying degrees 
of structure during different parts of an interviev;. 



Need for 
Specific 
Obj ect ives 



The importance of specific objectives in the interviev; is 
f req uent ly stressed . 

COMCLUSION 



^^When the obfjeotive is vague and ill-defined^ 
and no particular form has been given to the 
intcrvip:.\ . .the reliability can be shovm to 
he very poor and it follows that the vo.Hd.ity^ 
toOy would be very poor.^^ (research: Younge^ 

1956 : 26 ) 



Stress Another variety of interview discussed in the literature 

Interviews on interviewing is the stress interviev; . Mhen the job 

for which the applicant is being interviewed involves 
stress of some kind, the interviewer may attempt to simu- 
late the tensions of the position in order to observe the 
applicant’s emotional stabilit; and frustration tolerance 
under strain . While current government practices ^nd 
literature on school interviev/ing indicate that stress 
interviews are being used, research does not yet prove 
tlieir success — beyond the satisfaction expressed by people 
who have tried it (Hallows and Estep, 1957; School 
Management , 1964) . 



O 

ERIC 



119 



106 



Harly CAUTION 

Commitment 



Some intevvieweVB believe that in only a 
few minutes^ five to ten at the mo.^t^ an 
opinion can be formed to screen out 
obviously unqualified candidates. (research: 
FeaVj 1958) However^ research has not sub- 
stantiated the reliability of such an opinion 
for a wide variety of interviewers involved, 
in selecting teachers. 



In fact, the early part of the interview is when many 
interviewers commit themselves to a decision regarding 
the applicant. This commitment may be made either con- 
sciously or unconsciously. Bolster and Springbett (1961) 
found this same early commitment, although it was followed 
by tendencies to shift decisions according to these 
principles : 

• The amount of interview commitment at the time 
of a decision shift and the weight of the 
challenging information both affect the 
amount of change made in decisions. 

• It is easier to shift a rating in the direction 
of a rejection decision that in the direction 
of acceptance. 

• People have different amounts of sensitivity 
to negative evidence. 

• An item of information received toward the end 
of the interview carries more weight than it 
would if received earlier in the interview. 

CONCLUSION 



Therefore^ when information is received^ as 
well as its content:^ affects how it is per- 
ceived. (research: Bolster and Springbett y 
I961y and others) 



Use of 

Written 

Information 



If written information about an applicant is read prior 
to the interview, it is done to help structure the inter- 
view and to probe areas that should provide maximum 
information. Reading the information during the inter- 
view is justified on the basis of inability to acquire the 
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Information aliead of time and a desire to see it prior 
to leaving the individual, Wlien interviews arc con- 
ducted prior to reading written information , the ration- 
ale is usually that it prevents tlie interviewer from 
establishing biases, 

NOTE 



!lo genevalizations can he made regarding th'. 
goodness of interviews. This is due to the 
fact that the interviewer is a part of the 
measuring instrmient as he fudges each eon- 
didate; hence there is no common mcasuri-ng 
instrument to be studied^ yior can general- 
izatioy^s he made regarding the precision 
of the instrument, (viewpoint: Springhett^ 

195S) 



Functions 
of Inter- 
viewer 



Sources of 
Error 



To further complicate matters , the interviewer performs 
three separate information processes : gathering , pro- 

cessing , and evaluating information , Tliese processes 
are , to some extent , independent. Some insist 
(Springbett, 1958) that the same person need not do them 
all , and that they do not necessarily have to be done 
during the interview. The interview itself may ser\*e 
only to gather information from which another person 
makes the decisions involving synthesis and evaluation. 

In addition to gathering , processing , and eva] uating 
information, an interviewer may be called upon to predict 
what the applicant will do in the future. The two tasks 
of observation and prediction may have little relation 
to each other; some interviewers may be able to do one 
effectively and not the other (Giedt, 1955). 

Additional sources of interviewer errors are given in the 
literature.* Many of them result from attending to cues 
that are of little consequence , 



*See the discussion on "Factors Influencing the Interpre- 
tation of Data" on pps, 120-5 for a more general discussion 
of errors in the selection process. 
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CAUTION 






Administrators should be aware of tl^e fact 
tliat relatively minor items may influence 
the interviewer unduly. (research: Mandell^ 



1961b) 



Such items as the chance of a superficial resemblance of 
the applicant to someone the interviewer has known in 
the past , the interviewer * s reaction to nervousness on 
the part of the applicant, and the interviewer’s fear of 
weakness in the applicant are sources of error related to 
the interviewer’s attitude and expectations of the 
respondent (Mandell, 1961b). 

Often interviewer decisions are most nearly predictable 
from the interviewer’s action than from the actions of 
the applicant (Ulrich and Trumbo, 1965). This is the 
case when an interviewer projects characteristics of his 
own onto the applicant. For example, the interviewer may 
see the applicant as an unusually critical person when, 
in fact, it is his own attitude that is overly critical. 
In addition Sydiaha (1962) found that the tendency to 
empathize with some interviewees interfered with the 
obj ecti vity of interview decisions . 

Six common errors in the use of questions in the inter- 
view are identified by Magee (1962) : 

• stating questions so they can be answered by 
"yes" or "no," which elicits so little informa- 
tion from the applicant that hidden strengths 
and weaknesses are not revealed 

• unimaginative questions for which the astute 
applicant already has ready-made answers 

• leading questions which suggest proper answers 

• questions and comments which are non-neutral and 
reveal the interviewer’ s attitude 

o questions not related to the task at hand 

• questions that are already answered on the 
application form or resume. 
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c> 



The pernona.l can help cHsorinii.natc 

between applicants as h^ell as increase 
Qonfideyiae in the selection decision (research : 
Bolton^ 1QG8) 



Increased 
Discrimi- 
nation and 
Conf idencp 



Recent research (Bolton, 1968) has indicated that the 
interviewer has a greater feeling of certainty about his 
decisions when he has both the opportunity to see rand to 
hear t..e applicant. Audio-interview information increases 
the interviewer's ability to discriminate between appli- 
cants, Thus the personal interview, since it combines 
visual and audio information, contributes to the selection 
process by increasing the success and confidence with 
which people select from among a group of applicants. 



Summary, In summary, the validity of the interview for decision 

Interviev/s mailing is frequently challenged in the literature. There 

is no evidence that all interviewers can function equally 
well with all types of respondents, and there is a need 
to determine the value of the interview separate and 
apart from the personality of the interviewer, 

SUGGESTION 



AAditional research should he d.one to ayialyze 
the contributions of the intervieio ^ per se^ 
without the use of ancillary data such as 
application foms^ college records^ and grade 
point averages. (vieiopoint: research staff) 



Conclusion The following conclusions and recommendations should be 

and Recom- kept in mind: 

mendat ions 

• The interview may be valid or not according to 
the skill of the interviewer, the situation, 
and the nature of the intervievjees . The 
assumption that the interviewer , regardless 
of who he is or whom he is interviewing , is 
consistently able to gather , weight , and 
evaluate information accurately has not been 
proved. 

0 Interviewers involved in the select 1 on of 
teachers should receive training in the 
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inteiview process, since training has been 
demonstrated effective in preventing common 
interviewer errors and in improving the 
ability to discriminate among applicants, 

• More research followed by rapid and tdiorough 
dissemination of its findings needs to take 
place. Exactly what part does the interview 
play in the selection decision? I^at are 
the precise effects of different training 
programs for interviewers? How many inter- 
views are necessary for the best decision 

by both the interviewee and the school 
district? How should information gathered 
during interviews be recorded? l^o makes a 
good interviewer? Research on these questions, 
and on many more like them needs to be done, 

• I^ich intervie wers are the mo it succcessful? 

One large gap in the prai:tices of many school 
districts and industries is the failure to 
follow the future careers of applicants, both 
those selected and those raj ected, to discover 
the quality of the decisions that were made. 
People who tend to make good interview decisions 
should be recognized, and the fact that not 
everyone does should be a matter of concern 

and action for personnel directors. Continuous , 
local research of this nature should be a part 
of every selection program. 



Paper and Pencil Tests 

Purposes of Psychological and educational tests are used in the 

Tests selection of personnel because it is generally accepted 

that they supply information which gives additional and 
valuable assistance in making decisions about people. 

Many businesses use psychological tests in their data 
collection process. Business and industry are aware of 
the limitations and effect which testing programs have 
on their organizations, but the merit of the extensive us 
of tests for selection purposes in businesses cannot be 
denied because of their many successes (Ward, 1960), 
However, the contribution these tests make to the selec- 
tion process for school districts is unclear. 
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N’OTl' 



AlthongJi yt)]jcholoai (\il tcntn ca>: riivvl'/ 
pvc^H^t'v^ infomation not othcv'.n.[\<^ 
availal'lc ‘rnovi oihcv oata co'^ Icctior, 
i^'^^trnmentSy the Infomation from tnetn 
is not loiivevsaVlv jn'*ccJiot{vc. Therefore^ 
tests rnust he loQalhj valr'. dated. (research: 

syy^thesis) 



Research in industrial psychology can indicate v/hich tests 
predict specific kinds of success; where this is done, the 
use of tests can simplify the selection of employees. 
However, school administrators may re-jcct paper and pencil 
tests on the ground that human behavior is too complex to 
he handled by other than implicit, intuitive means 
(Stone and Kendall, 1956). Although thi.s reluctance to 
use tests may be attributed to a number of factors (e.g., 
cost, lack of trained personnel, fear of dehumanization), 
and although tests may not he absolute and perfect pre- 
dictors, it is clear that they do have the capacity to im- 
prove batting averages in selection (French, J. , 1956) and 
thus should he examined for their utility in specific 
school situations . 

For example , if school district requires that teachers 
have a specific level of intelligence and/or a special 
aptitude, testing may be the only V7ay to get oh-jective 
and quantifiable information. The reason, local validation 
is necessary is because some of the standardized tests 
have not been effective as predictors in certain condi- 
tions. For example, a study with student teachers indi- 
cated that only selected scales of the Minnesota Multi- 
phasic Personality Inventory appeared to be effective as 
screening devices at the preservice level, while tests of 
academic acliievement and intelligence (the School and 
College Ability Test, the Purdue Intelligence Test, and 
Sequence Tests of Educational Progress) did not prove 
to be predictive as selection devices (Mascho, 1966). 

NOTE 



What test users in an employment office are trying 
to predict with tests is the potential poh success 
of the applicant^ not his intelligencf ^ mechc.ni cal 
ability or his personality. (view^ro'h'it : Alhr'^bht^ 
Glenror> and Smithy 1963:18) 



If a decision maker believes that there is a need to con- 
sistently apply standardized procedures to all applicants 
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in order to assure that the best selection wilJ be made, 
then he cannot disregard the value of tests. Tests, in 
themselves, are neither good nor bad and should he se- 
lected in terms of the appropriateness for a given situ- 
ation . 

CAUTION 



Paper and pencil tests must have a proven vela- 
tionship to Jo!) success^ otherwise hiring high 
scorers only amounts to hiring those wiio are good 
test pe2*fo7viers . ( research: synthesis — Cron- 

bach and Gleser^ 1965; Guion^ 2965; Albright^ 
Gleymon ayid Smithy 1965) 



What Tests One useful way to look at v^7llnt tests measure can be seen 

Measure in Figure 3.5. This "model” of the measurement of human 

characteristics is somewhat generalized but useful in 
seeing the continuum of characteristics to be measured 
and the kinds of precision one can expect from psycholo- 
gical, physical, and educational tests. 

Appreciation of the worth of competently used tests and 
understanding the need for local validity are not the only 
factors which influence the decision to use tests for 
selection and placement. 

NOTE 



Factors which nmst be considered in the decisi.oy^ 
to use psychological tests for selectioyi pur- 
poses include: 

0 ultimate worth iri predicting applicant 
success 

0 cost 

0 impact on potential applicayits 

0 current employee morale 
( viewpoint : research staff) 



A testing program is not inexpensive. Development of 
materials, training of personnel and informing the staff 
are costly. Employee-employer relationships and public 
relations can be adversely affected if the purpose and 
procedures of the testing program are not clear to staff 
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height: 



Physical 
Charac^CL.•is ti cs 



Abilities and 
Skills 

Interests 



Personality 

Traits 



weigh t 

visual acuity 
hearing 
dexterity 
math, ability 
verbal intelligence 
clerical skills 
mechanical aptitude 
mechanical interest 
scientific interest 
economic interests 
cultural interests 
sociability 
dominance 
cooperativeness 
tolerance 

emotional stability 



More 

Measurement 

Precision 



1 r 

Less 

Measurement 

Precision 



figure 3.5. Measurement of Human Characteristics . 
(Albright, Glennon, Smith, 1963: p. 41) 
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OClier Uses 
of Tests 



Resources 
on Tests 



I^iat are 
They? 



and cnmmunity. Die question of their value*, vnlidiLv imt- 
withstandlng, must be dcaJt with, i.e., one imr-,t /ilwa'.n; 
answer the question of whether the improved predict inn of 
success or failure is worth the cost. 

'tests for prediction purposes, of course, are not limited 
to their use in selection and placement alone. They may 
be used in supervision to discover the worth of in-service 
experiences, i,e., to determine wliat in-service training 
to give to whom; they al.so may be used to establish crite- 
ria for success and resulting position description. In 
the latter case, they may be used to verify the standards 
of success that a school district desires. (For example, 
if a school district desires intelligent teachers, tests 
could discover if intelligence really does relate to good 
teaching.) There is a definite need for sciiool districts 
to consider these uses of tests in local research efforts. 

School districts considering a testing program for select i 
purposes would find two resources quite valuable: 

Albright, Lewis F, , J. R. Glennon, and Wallace J. 
Smith. The Use of Psychological Tests in Industry . 
Howard Allen, Tnc., 1963. 

Cronbach, Lee J. , and Goldine C. Gleset: Psycholog - 

ical Tests and Personnel Decisions . University of 
Illinois Press, Urbana, 1965. Second Fdition. 



Assessment Centers 



A relatively recent development in gathering selection 
information is the use of actual performance assessment. 

In business and industry, this has taken a variety of 
forms, from centralized assessment centers to decentral- 
ized on-the-job performance assessment. Centralized 
assessment centers are usually conference areas with one- 
way observation windows where the candidates do the 
required exercises, whereas decentralized on-the-job 
performance assessment usually occurs in selected regional 
or branch offices. Because of the expense of operation, 
assessment centers are primarily used for selection of 
managers from employees. However, they can be used for 
initial selection into an organization, and they offer 
promise for use in teacher training institutions and 
large school districts. 
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Nori:; 



The assessneyit center approach^ oomhinn^na 
psij ohological teste ^ simulation exercises ^ 
and observation of group behavior is a 
relavively recent development i.n the 
appraisal of a candidate's potential for 
success and is gaining acceptance. 

( practi ce : industry) 



At the assessment center, the candidates may participate 
in: 

• a compreiiensive interview 

• several paper and pencil tests of mental ability 
and knowledge 

• job simulation via an ^in-basket" exercise 

• leader less group discussions 

Each candidate's performance is analyzed by trained 
personnel and summarized in a detailed written report 
v/hicli gives an estimate of his potential for specific 
positions. Some companies have used a decentralized 
approach where the actual employnient station becomes 
the center of assessment. Results of studies consis- 
tently show that assessment centC'.r evaluations are good 
predictors of success (Campbell and Bray, 1967). 

Although school districts have net yet used the kind of 
performance appraisal embodied in the assessment center 
approach , these methods are being discussed . Likewise, 
other approaches to performance assessment via teacher 
internship , micro teaching , and recently combined efforts 
of training institutions with local districts and pro- 
fessional associations are occurring and show promise of 
benefit. The emphasis of these activities has been on 
teacher training rather than on selection, but they do 
contain components which relate strongly to the selection 
process . 



and criterion measures which contributed to the selection 
process; 91% of the men selected under the assessment 
center approach were promoted. Other studies have indica- 
ted that the difference the assessment center makes in 



Results and 
Benefits 



Several studies by Midwestern and New England companies 
(Campbell and Bray, 1967 ) identified assessment ratings 
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selection policy is related to its acceptance by the 
management of the organization. Supervisors must be 
encouraged to attend , and even participate as staff, In 
the assessment centers. This opportunity to evaluate 
the diagnostic information the center makes available 
permits greater involvement in the selection process 
and develops evaluation skills in those wlio participate 
as staff A follow-up on persons assessed in centers 
can determine the value of the process, and can determine 
whether modifying the process and/or training tliose wlio 
function within it is necessary. 

Processing Candidate Information 

Data collection provides a basis for selecting or reject- 
ing candidates , Therefore , the information that is 
collected must be processed so that it actually does 
serve that purpose. Several options are available for 
the processing of information; all are related to the 
degree of sophistication and precision of the total 
process , 

CONCLUSION 



Where data ooZZeotion becomes move pveci^se 
and objeotive^ aj%d when the information is 
used in quantitative and composite form for 
prediction^ the processing of that in forma- 
tion becomes more compZex. (viewpoint: 
research staff) 



The most common technique for processing information on 
a candidate involves tlie creation of a file folder, 
containing his documents and miscellaneous selection 
information. The file may be reviewed by the personnel 
officer and/or principal, and may be assigned a reject, 
hold, or hire rating. 

When districts can determine which information best pre- 
dicts success by performing local validation studies, 
information processing becomes more complex. Data must 
somehow be condensed and coded, so that candidates can 
be ranked when the point of final selection approaches; 
the processing of the information and the resulting 
format play a significant role in that final decision. 
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Research indicates that the format in which i nf<-)rma t i on is 
presented enhances selection decisions, particularly 
when data is condensed on a single summary document 
rather than left as a mass of data in a file folder 
(Bolton, 1969). When school districts move tov^ard more 
sophisticated and precise selection procedures , using 
large amounts of data to validate the selection process 
as well as to make actual selection decisions, the need 
for computer service to handle this information becomes 
more apparent , 

Computers have considerable potential use in the record- 
ing, processing, and retrieval of candidate data in the 
selection process. However, because of cost, cooperative 
intradistrict planning and purchase of computer time may 
be the only way smaller districts can have computer 
service available , 



Approaches 
for Smaller 
Districts 



It may also be that computer service is not necessary for 
the smaller district and their selection efforts, if 
sufficient attention is paid to efficient processing of 
paperwork. For example, visible record files which show 
at a glance the number of applicants for specific jobs, 
the comparison of candidates, and what action needs to 
be taken to make the selection process more efficient 
(Bolton, 1962 ) , The information processing system can 
also be used to develop a graphic record v/hich shows when 
most applicants inquire, send in the application, and are 
interviewed (Bolton, 1963). This can guide the timing in 
making actual selections , 



Rationale 
for Sequence 



Seque n ce of Data Collection 

Determining the sequential use of data collection instru- 
ments is based on strategy of decision making and the 
contribution each ^’instrument*' makes to the selection 
process . 



SUGGESTION 



Those instvwnents which provide a rough 
screening of applicants should he used, w,th 
the larger applicant pool; those that 
differentiate more precisely between final 
candidates can be used durijig the last 
stages of selection. (position: research 

staff) 
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It is possible that school districts may interview an 
applicant before his application is reviewed or may 
delay paper and pencil tests until all other data are 
collected. However, the basis for establishing these 
sequences would be unclear unless information on the 
local validity of these procedures is available. 

For example, a school system may find that a locally 
tested and proven application form does the best rough 
screening and that the best predictor for differentia- 
tion between highly qualified candidates is the perfor- 
mance assessment approach. If this is the case, then a 
data collection sequence may be established which 
reflects this information. 



Prerequisite Of course, if combinations of instruments do both rough 
to Sequence and precise differentiation and can be administered 

rather easily, the sequence decision may be less one of 
deciding on sequential use of individual instruments than 
one of acquiring the proper combinations of data collec- 
tion devices to use. When lo.cal districts develop 
validated (proven) predictors and seek that information 
on new candidates, the establishment of a functional 
sequence can become clearer to them. 



Bases for 
Sequence 



To summarize, the decisions about sequence of data collec- 
tion are based upon these factors: 

o the validity of each instrument (application form, 
interviews, etc,) used to collect .Information to 
predict success 

• the degree (gross to precise) of candidate differen- 
tiation each instrument provides 

• the specific information each instrument contributes 

• the particular selection purposes instruments serve^ 
(e,g,, an instrument which supplies data used for 
early elimination of applicants may be used in a 
different position in the sequence than information 
about strengths and weaknesses to be considered in 
later supervision) . 

Candidates may, of course, exhibit differences which may 
be ascertained better by a flexible sequence. The sequence 
of data collection will have direct bearing upon the 
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General 

Influences 
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selection decision’s goodness or validity. More ossen- 
tinl, however, is the notion that sequence provides tiu’ 
framework for management of the information used to 
make these decisions. 



Making Decisions 

Factors Influencing the Interpretation of Data 

In the research literature concerning the selection of 
teachers, few clues are given which suggest how data 
which describes teachers is to be interpreted . However , 
information is available concerning what influences the 
interpretation of data used in making the selection 
decision. Discussions regarding these influences can be 
categorized as f ollovjs : 

• general influences and considerations 

• factors which enhance one’s ability to interpret 
data correctly 

Q factors vjhich diminish one ’s ability to int erpret 
data correctly. 

An administrator needs to be aware of those factors or con' 
ditions v/hich v;ill have an impact on his ability to inter- 
pret data used in the selection decision or in his choice 
of desired teacher behaviors. Such an awareness is the 
beginning point for an examination of those factors and 
conditions (Guetzkow and Forehand , 1961) . For example , 
Bolton’s study (1968) demonstrated that in the teacher 
selection process, the format of information provided to 
a decision maker affects his ability to discriminate, 
rate consistently, feel certain about his decision, and 
efficiently use time required to make the selection 
decisi on . 
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SUGGESTION 



The selection decision is improved by: 

(a) using a single page summary document y 
rather than multiple documents descriptive 
of an applicant; (b) providing instructions 
to the decision maker regarding how to 
process information; and (c) using interview 
information in audiovisual form rather than 
audio form only or providing no interview) 
information, (research: Boltony 1968) 



School personnel should resist the tendency to confront 
selection decision makers with too much information 
and reduce data to that which is needed and usable. 

SUGGESTION 



Substantial effort should be made to deter- 
mine what information is needed locally in 
the selection process; a substantial 
reduction may be made in the amount of 
information with which decision makers are 
confronted* (research: Hickey^ 1970) 



The training and experience of the person making selection 
decisions appears to influence the way he interprets 
information. For example, Halpern (1955) concludes that 
individuals can recognize most readily those behaviors 
and characteristics in others that are similar or equiva- 
lent to the characteristics and behaviors which they 
themselves possess. Therefore, a potent influence on the 
interpretation of data is the background and training of 
the person doing the interpretation. 



Some Factors 
Enhancing 
Interpreta- 
tions 



Some of the more interesting questions regarding inter- 
pretation of data relate to a supposed difference between 
men and women in their ability to interpret data or make 
j udgments according to data being considered . In this 
regard, there appears to be no sex difference in the 
ability to interpret data; however, some reports show that 
women have a slightly superior ability to judge (Taft, 
1955). Explanations for this slight difference between 
sexes are incomplete, and one seems safe to conclude that 
any differences in interpretation and judgment which might 
exist are inconsequential. 
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While we may assume that: a difference in sex wilJ nol 
enhance the ability to interpret data, Taft (1955) has 
shown that individuals who possess the ability to remain 
socially detached make more accurate judgments than those 
who are not so detached , 






Many administrators are beginning to realize that th.oir 
selection decisions are altered by a host of influences 
on their ability to interpret data correctly; however, 
few are recognizing steps which they can take to enhance 
their chances of making more valid selection decisions. 
As an example of a step which might be taken, one could 
insure that evaluations are based on the judgments of 
multiple raters rather than single raters since research 
has shown that when multiple raters are used in making 
evaluations, the information generally will be a better 
indication of teaching behavior than if single-rater 
evaluations have been used. 

CONCLUSION 



Multiple raters enhance predictions of 
teaching success over shnjgle raters. 

( research: Lawler y 1967 ) 



Advice regarding factors which will improve the selection 
decision abound in the literature even though this advice 
often does not seem to be based on specific research 
findings. However, two points appear useful. The first 
point suggests that the selection decision is improved 
with an increase in the number of available candidates. 

CONCLUSION 



The wider the degree of choice in the selec- 
tion decision^ the better the chance that 
the selection decision will he valid, 
(viewpoint: LopeZy 1965) 



The second point is concerned with the importance of 
professional training and experience as the basis for 
making sound selection decisions. 
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NOTE 



Professional knoialed^e has proved to he 
one of the more successful predictors of 
teaching performance. (research: Flanders, 



1969 ) 



Factors 

Causing 

Errors 



Errors by 
Persons 



In addition to the above statements affecting interpre- 
tation of data, it appears that some people "interpret” 
data primarily by looking for factors that eliminate 
applicants. However, as Stone and Kendall (1956) point 
out, an identification of the absence of disqualifying 
factors in an applicant is not equivalent to predicting 
his success. It seems that one would be wise to consider 
the often-made statement : It is ex tremely easy — and 

costly — to let one's desire for new information influence 
a decision about the appropriateness of that information. 

Considerable attention is given in the literature to 
categorization of influences which may cause errors in 
the interpretation of data used in the selection decision. 
These errors may be grouped under the following two 
general headings: (a) errors committed by persons in the 

selection process, and (b) errors committed by organ: 'na- 
tions or institutions in the selection process. 

Persons involved in the selection process may be suscep- 
tible to one or more of the following errors; consequently, 
a conscious effort should be made to minimize their 
occurrence. 



0 leniency 



halo effect 



• central tendency 
o logical 



-bias which allows one to reduce 
the importance of negative 
characteristics exhibited by 
a candidate 

-bias in favor of an individual 
candidate resulting from an 
overemphasis placed on specific 
desired characteristics 

-the tendency of individuals to 
average judgments 

-ambiguous interpretation of 
similar characteristics, e.g., 
integrity and truthfulness 
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• contrast 
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types 


of 
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faulty weighting 



confounding 
prenature closure 



t imes 

--tendency of professionals to 
inappropriately weight some 
factors used in making pre- 
dictions 

— simultaneous consideration of 
more than one characteristic 

— distortion of subsequent infor- 
mation because of early con- 
clusions 



Errors by Organization s involved functionally in the selection 

Organizations process should develop procedures which will help minimize 
the following comiaon organizational errors: 

• careless treatment of candidates 

• hyperdependence on expert opinions 

• misjudging job requirements 

o making stop-gap appointments 

• stereotyping prospective candidates 

• disregarding the "organizational" personality 

9 overlooking personal compatibility with personnel 
within the organization 

Suggested There is little research available which indicates how 

Actions data descriptive of teachers should be interpreted. 

However, the analysis of factors which enhance interpre- 
tation and factors which cause errors suggest specific 
actions which offer promise f or reducing probl ems of data 
interpretation . These actions are : 

• limit the number of behaviors that are to be 
assessed in the selection process 
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• develop specific definitions of desired character- 
Lstics 

• specify behaviors desired and situations wherein 
those behaviors are appropriate (for example , 
consider the requirements of an urban classroom 
filled with alienated students as onposed to a 
suburban classroom filled mainly with students 
motivat ed for school activities) 

• focus on specifics rather than general impressions; 
where possible avoid inferences 

• assess observations immediately, rather than delay 
assessment 

• assess behaviors independently (for example, teacher 
behaviors, pupil responses, and pupil-teacher 
interactions) 

0 suppress rating and observer biases based on an 
awareness of those biases 

0 replicate observations and assessments. 

Types of Decisions 

When individuals involved in the teacher selection deci- 
sion confuse individual decisions with institutional 
ones, difficulties arise. Cronbach and Gleser (1965) 
suggest that selection personnel xust be aware that they 
are making institutional, not individual , decisions. 
Institutional decisions use a common value system for a 
large number of comparable decisions in an attempt to 
maximize organizational gains *'in the long run.** 

However, an individual decision may be confronted only 
once; consequently, a person’s value system at that time 
is used to make the decision. Since the individual may 
never have the opportunity to maximize gains over a large 
number of comparable decisions, his decisions are more 
subj ec t to changes in his value system. 

Cronbach and Gleser (1965) identify three vital elements 
essential to institutional decision making: 

0 a clear statement of desired outcomes provided by 
spokesmen for society 
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o a clear statement of empirical predictor-trentmcMU- 
outcome relations provided by research 

t a rational combination of these two into an optimal 
decision rule, 

Regcirdless of the institutional or individual classifi- 
cation of decisions made in the selection process, it 
seems imperative in the interest of sound personnel 
practices that selection decisions be made according to 
organizational requirements and not the personal prefer- 
ences of those involved in the process. 



Decision 

Strategies 



Clinical 

Decisions 



Strategies 

When making the final selection decision, a person tends 
to use one of two different strateiges. 

• Subj ect ive. At one extreme are those authorities 
who recommend what is essentially an intuitive 
judgment, called "clinical" by Meehl (1954:3). 

• Obj ective . Others insist that only an obj ective 
process using the quantification of verifiable 
information can render valid decisions. The 
objective method is sometimes referred to as 
"actuarial" or "statistical" decision making. 

In general, opinions still differ about which of these 
two strategies produces the sounder judgment. Authori- 
ties do agree that an essential task of the decision maker 
is to predict the future performance of the applicant . 

Hota^ this prediction can most reliably be made is the 
iscue. Scott (1964:93), discussing managerial selection 
decisions, asks 

...whether or not a selection decision maker 
utilizing clinical methods will be able to 
predict managerial performance more accurately 
than one who approaches the task with tests and 
other psychometric data that have been corre- 
lated with measures of managerial performance. 

" Clinical " decision making refers to the use of processes 
resembling those of the psychologist (who regards each 
person as a unique individual with a set of traits, 
abilities , motives , and values so entirely different from 
all others that he can not be adequately described by a 
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set of standardized tests and procedures) (Scott, 196A). 
The use of intuition that cannot be supported by statis- 
tical evidence is considered necessary because the 
uniqueness of each individual does not allow for previous 
development of measures suited to his particular pattern 
or structure of behavior (Scott, 1964). The interviewer * s 
understanding and empathy may play a larger role in this 
kind of decision than does objective evidence (Sydiaha, 
1962). 

What does determine the validity of subjective methods? 
Other than counting the number of "hits or misses" after 
the fact, there exists no means to m'iasure success. 

Mandell (1964:27) states: "We cannot discuss the validity 

of subjective methods in general; their validity is as 
high or as low as the competence of the evaluator." 
Obviously , considerable training and experience are 
necessary before one is competent to make this kind of 
selection decision (Hammond , Hursch , and Todd , 1964) . 

On the other hand, objective or actuarial methods are 
supported by those who believe that, complicated though 
behavior patterns are, behavioral dimensions do exist 
in common in individuals and provide a sounder and more 
explicit basis for selection than intuitive decisions do 
(McMurray, 1947; Scott, 1964). Particularly where the 
job environment is relatively standardized and there are 
large numbers of repetitive decisions to be made, 
attempts to develop objective instruments and methods 
should be considered (Mahoney and England, 1965). 

CONCLUSION 



Most current advocates of decision-’inaking 
strategies see the necessity for a combi- 
nation of clinicol and actuarial methods, 
(position: Meehl^ 1954; Scotty 1964; 

Bellows and Estep ^ 1957; OSS^ 1948) 



Several authors (Bellows and Estep, 1957; Sydiaha, 1962) 
who recognize the advantages and disadvantages of both 
methods recommend a combination of the two to arrive at 
the best selection decision. The choice of strategy 
for processing decisions then becomes a matter of degree 
and less a matter of either clinical or actuarial 
selection decisions. As much as possible, item analysis 
and validation of information should be used, with reliance 
on properly trained clinical "intuition" when and where it 
is necessary. 



A Matter 
of Degree 



Actuarial 

Decisions 
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Other 
Strategy 
Issues 

• the use of eliminator or selector variables 

0 the use of a probationary period as an extension 
of the selection process 

0 which variables compensate for one another 

0 who is to be involved in various stages of the 
process 

0 the sequence of activities 
0 placement procedures 

0 the metliod and timing for making offers of employment. 
NOTE 



As one excovines the variables being used as 
vrexliotors^ he should choose those to he used 
to eliminate certain applicants: those which 
will he used as selectors; and those to be 
used both as eliminators and selectors. 



In considering strategics for making selection decisions ^ 
issues and concerns other than the actuarial -clinical 
controversy are significant. Of particular importance 
are sucli items as: 



Eliminators For example, one might decide that prior experience with 

Versus youth activities would make an applicant more suitable, 

Selectors but that lack of such experience would not eliminate an 

applicant; that habitual absenteeism in prior work 
experience would be sufficient to eliminate an applicant; 
and that scholarship migh t , depending on its quality , 
cause an applicant to be selected or eliminated . 

0 The eliminators function as hurdles which must 
be passed. 

0 The selectors are those variables which indicate 
what desirable behaviors will be exhibited. 

Where eliminators are used , one should be cer tain that the 
predicted behaviors (for example, ”morali ty compatible 
with the district," or "lack of absenteeism") are actually 
necessary to the teaclier ’ s satisfactory performance . 
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NOTE 



In addition^ one should he relatively oer^tain 
that the behaviors are necessary for acceptable 
initial work and are not likely to be attained 
after the position is taken. This latter 
point is especially crucial when the niMvher 
selected per the number who apply is high^ 
because otherwise acceptable applicants who 
could acquire needed skills on the dob may be 
eliminated. ( viewpoint : research staff) 



For example, a certain position may require a given level 
of organized activity in the classroom. If evidence from 
practice teaching is used, a given applicant’s credentials 
may disqualify him by indicating that activities during 
practice teaching were not very well organized. An error 
might be made in immediately eliminating such an appli - 
cant . Further inquiry may indicate that this applicant’s 
attitude toward organized activity is excellent, but that 
his skills in planning, allocating and sequencing are 
weak. If he has had very little assistance x^7ith these 
skills, he may respond favorably to a helpful principal 
and colleagues who serve as good models. Note that the 
behavior required was not changed; but that a decision 
was made that the behavior was not necessary prior to 
selection for the position. 

NOTE 



JIhen the selection period is considered to 
extend into the period of employment in the 
form of a probatioyixiry period^ decision 
strategies are not likely to focus on vari- 
ables which eliminate applicants; rather ^ 
the focus is more likely to be on variables 
which function as selectors, (practice: 
schools^ industry ) 



Use of Where the possibility for release of a teacher is low, as 

Probationary is the case where a probationary period does not exist. 
Period the decision makers are less likely to risk hiring 

individuals who have some evidence of negative information. 
Therefore, the decision maker tends to use a strategy 
which emphasizes information likely to eliminate an 
applicant than to attend primarily to selector variables. 
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Compensa- 
t: ion 



Prior t:o making sc.lGction decisions, one should ascertain 
whether predictor variables interact so as to compeiisate 
for one another. For example, does intelligence compen- 
sate for liard work or diligence, or vice versa; would 
experience substitute for education? \>Jhere compensation 
occurs, individuals assessed on a number of dimensions 
may differ quantitatively on several dimensions and 
still have an overall prediction of similar value to the 
organization. 



Personnel 
Involved ; 
Sequence o 
Activities 



The type of personnel involved 
and the sequence of activities 
f district to district and among 

NOTE 



in the decision process 
varies considerably from 
industrial organizations. 



The sequence of activities for information 
collection and decision making is affected, 
by which phases of the selection processes 
are centralized. (practice: schools) 



The two sequences which follow are examples of divergent 
practices representing different emphases on central 
control of information collection and assignment of 
authority for making choices. 

Sequence //I ; Strong Principal Involvement 

a. central office conducts recruitment 

b. central office conducts screening interview 

c. central office processes written records 

d. principals examine written records, information 
regarding initial interview 

e. principal is responsible for contacting appli- 
cants in whom he is interested 



f. principal interviews 

g. central office interviews if the principal’s 
interview is favorable 



h. 



o tlier principals 
to the applicant 



who have vacancies of interest 
interview the applicant 
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Placement 

Procedures 



i, principal(s) make recommendation that offer(s) 
be made 

j, central office makes offer(s) 

k, individual decides which offer, if any, he 
will accept 

Sequence //2 : Strong Centralization 

a. central office conducts recruitment 

b. central office processes written records 

c. central office conducts interview 

d. central office summarizes information from 

written records and interview 

e. written information and summary are routed 

to all principals who have potential vacancies 

f. principals make recommendations regarding 
selection 

g. central office makes final decision 

The functions of central office and building personnel are 
different. Central office personnel are responsible for 
coordinating the total system; consequently, when uniform*- 
ity is desired, they must initiate procedures that will 
produce it. However, building personnel such as principals 
are in a much better position to explain the individual 
problems and diversity among their subunits and to relate 
to the personal concerns of the applicants. 

NOTE 



^/hen a 'person is hired for a specific position, 
no separate strategy for placement is necessary ; 
hut when teachers are hired into a "pool, " 
special placement strategies are necessary, 
(practice: schools) 



At least four strategies are currently used to make assign- 
ments from a pool of teachers. 
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a Central office makes the placement decision. 

• Central office makes the placement decision on 
the approval of the principal, 

• A committee makes the placement decision. 

9 The applicant has the opportunity to make a 
choice and be approved by a principal who 
has a vacancy* 

^ CAUTION 



All procedures regarding making job offers 
involve the risk of losing an applicant. 
Locally -conducted research is needed to 
deteimine which procedures are likely to 
he detrimental to the organization, 
(practice : schools^ uuiustry) 



Job-Offer The following points should be considered before a job 

Strategies offer is made: 

• Are better applicants likely to apply within 
the time period when an offer should be made? 

o Should multiple offers be made? 

• Who will decide when to make an offer? 



Procedures in use include: (a) making an offer at the 

time of the interview; (b) indicating to the applicant 
when the decision is likely to be made (and then establish- 
ing appropriate controls so the decision can be made 
within that time) ; (c) indicating the degree of interest 

that the district has in the applicant and then leaving 
the responsibility for checking back to the applicant ; 
and (d) asking the applicant when he must make a decision 
and adhering to that date. 



Cutting Scores 



Scores which are 
instruments (the 
in several ways, 
that is, as each 



derived from the various data collection 
application, interview, etc . ) can be used 
They can be used as individual hurdles ; 
data collection device is used, a cutting 
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score is set and a decision made to accept or reject. 

More commonly, a total composite for each candidate, 
based on all data collected, is used to establish a 
single cutoff score, 

A multiple cutoff method can be used as an indicator of 
success, where all scores are treated as continuous vari“ 
ables and a ’’profile" developed for each candidate. This 
method has several advantages. It does not assume that 
strength in one ability compensates for inadequacy in 
another important ability; and it retains the scores of 
separate tests in the record and permits more effective 
selection when dealing with marginal candidates because 
it is not an undifferentiated composite, but more or less 
a profile. 



Assessment of the Selection Process 



The Validation Process 



The outcomes of selection decisions must be subject to 
evaluation if the selection process is to be improved . 
Thus, if the selection decision is to be self -correcting , 
an evaluation system must be established which will 
determine whether a given district’s basis for selecting 
its teachers has been effective in selecting the kinds of 
teachers needed in a given situation. 

NOTE 



The ultimate validity of the selection process 
rests in how efficient it is in forecasting a 
teacher ^s success in meeting a given school 
district's needs* (viewpoint: Crissy^ 1952; 

French^ f/. , 1966; Gage^ 19^8; Thorndike^ 1949) 



Validation, in order to be useful, requires that objective 
investigations be conducted at the local level. The 
nature of the evidence gathered from these investigations 
is dependent upon the "type" of validity required. Many 
types of validity can be defined. Those commonly discussed 
in the literature are: 

• predictive — the extent to which a relation- 

ship exists between measures of 
desired behaviors and future 
behaviors which are exhibited 
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• concurrent 



— the degree to which test scores 
are associated wi th measur os of 
desired teacher beliaviors , 
assuming these measures and 
test scores are available con- 
currently 

• content — the degree to which a test 

includes a representative 
sample of all the tasks tliat 
could have been included 

• construct — a judgment that a test does in 

fact measure a specified 
attribute to the extent that 
it can be used in the predictio 
of behavior. 

Regardless of the ''type" of validity desired, it should 
be kept in mind that there is no such thing as general 
validity for any test since its usefulness as a selection 
instrument has to be proved for every job in every situa- 
tion in which it is used. Further, even when accurate 
forecasts are made, decision makers must recognize that 
working conditions and people change, and consequently, 
the obtained validity is likely to be only temporary. 

This makes it necessary to periodically check the 
validity. (Lawler, 1967; Bellows and Estep, 1954) 

CONCLUSION 



The selection process is dependent upon 
evaluative feedback resulting from local 
investiaatioyis which are designed to 
validate the process; such validation 
should be checked periodically because 
of changing conditions. (research and 
viewpoint: Bello^'^'S and Estep^ 1954; 

Hi?irichsj 1960; Redefer^ 1962; Eunnally^ 
1967; Lawler, 1967) 



Measures of Effectiveness 



The value of any meas 
(a) statements regard 
that can be made, (b) 
dures to be use 
cation of the r 



uring device can 
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operational def 
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of effectiveness as validity and reliability , (Ebel , 

1961) 

When little or no relationship is found between a measure 
which is used to predict teacher behaviors and those 
behaviors which the measure is supposed to predict, it 
is difficult to determine whether this is due to the 
fact that the measures are poorly chosen or because the 
desired behaviors are unpredictable due to unreliability 
of their measurement (Thorndike, 1949). This is par- 
ticularly significant in selection research since many 
measures of desired behaviors are based on judgments of 
people’s behavior. This is critical since it bar been 
found in many instances that there is a significant low 
consistency in the ability of people to judge others. 

Obviously, measures of effectiveness used in the selection 
process which are low in both validity and reliability 
are not beneficial for selection purposes. However, 
this does not imply that measures which have high coeffi- 
cients of validity and reliability will be usable, since 
for example, such measures may be very difficult to use. 

NOTE 



Important factors relating to the usability 
of measures include: 




ease of administration 


# 


ease of scoring 


0 


ease of interpretation and application 


0 


reasonable cost 


0 


proper construction 


(viewpoint: Cronhach and Gleser^ 1965) 


Analysis 


of Sources of Potential Errors 



^ Hyman (1954) contents that the ability to formally recog- 

■ nize errors marks an advanced stage of any science. If 

we accept this contention, then it becomes imperative that 

I 

1 
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an analytical scheme for catching errors be j-ncorporat eii 
into tlie selection process, ft is especially important 
that this analysis of error of prediction he present in 
all stages of the selection process, since only through 
such error analysis can control and effectiveness of the 
selection process be assured . 

SUGGESTION 

' 

It is of vital importance in the selection 
of teachers to conduct a. continual anoiUjsis 
of error in the following stages of the 
selection process: 

0 needs assessment 

0 criteria determiyiation 
recruitment 

0 collection and processing 
of data 

0 making predi ctions 

0 developim strateo'^* es 
0 fino.l selection 
0 plo.cement 

To accomplish this effectively^ an 
efficient y^ecord keeping system is 
necessary- (viewpoint: research staff) 



Summary 



Need The first step in the selection process is the determi- 

Determination nation of needs for teachers, both in quantity and kind. 

Needs should be determined in relation to the overall goals 
of the organization, the resources available to the dis- 
trict, the previous staff accomplishment, and previous mis- 
estimates . 



Criteria No single criterion exists to predict the *'good” teacher. 

Determination Instead, criteria are changing, multiple, and situation- 
ally determined. A position analysis is recommended and 
should include information about the j ob requirements and 
the employee characteristics . Examination of position 
analyses should be repeated frequently. Teachers, vice- 
principals, department heads , central office personnel 
should all be involved in the development of criteria. 

Recruitment The purpose of recruitment is to provide an applicant pool 

that is adequate in both quantity and quality, with suffi- 
ciently diverse characteristics to fill all needs. The 
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Collection 
of Data 



Processing 

Candidate 

Information 



cost of recruitment must be considered in relation to its 
results. Recruiting procedures should be more diversified 
than they generally are now and should be studied to deter- 
mine their effectiveness . 

Caution should be used in the utilization of letters, 
inquiries, and resume's for selection decisions, Corre- 
lational studies can be made locally to determine which 
information on application forms best predicts future 
success as measured in the district. Letters of recom- 
mendation are often biased and inaccurate unless they 
contain precise information. 

Interviews are used to collect information from the 
applicant as well as give information to him about the 
position. Interview practices vary widely in the amount 
of structure of the interview, the sequence of persons 
interviewir»g each applicant, and the amount of training 
given to interviewers. Industries tend to make more use 
of group interviews. Stress interviews are being used in 
some places. The goodness of the interview as a technique 
is determined by the skill of each interviewer, who is, in 
effect, a part of the "measuring instrument." Interview- 
ers make errors in being influenced by relatively minor 
items, committing themselves too early in the process, 
einpatnizing with the applicant, projecting their own 
characteristics onto interviewee, and asking inappro- 

priate ques’.Lons. The personal interview has the value 
in increasiiig the discrimination between applicants and 
the confidence felt about the selection decision. 

The value of written psychological and educational tests 
has been established, but needs to be locally validated. 
Business and industry tend to use tests more than schocls. 
Tests need to have proven relationships to job success , 
not just to the traits measured by the test. A good 
testing program increases the cost of selection decisions. 

An>sessment centers represent one of the more recent develop- 
ments in selection processes that is gaining acceptance, 
to date largely in industry. Studies show that assessment 
centers are good predictors of success on the job. 

The processing of the data collected becomes more complex 
as districts acquire more sophisticated information and 
use a composite form for the prediction of success. Data 
must be condensed and coded so candidates can be ranked 
for the final decision. A single summary document is 
recommended. Computers may be needed in larger districts 
to handle the increased paperwork of improved data collec- 
tion procedures. Only those instruments designed for rough 
screening should be used with the larger applicant pool. 
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Strategies Opinions differ about the need for subjective versus 

for Decisions objective selection decisions in the hiring of teachers. 

In reality, a combination of methods is usually necessary, 
and may even be desirable. In addition, it is important 
to decide which variables used as predictors of success 
will completely eliminate a candidate and which will be 
used as selectors. The use of probationary periods after 
employment allows more emphasis on selectors than elimi- 
nators • 



Interpre- 
tation of 
Data 



Some variables, such as hard work, may compensate for the 
lack of other qualities and should be considered. If 
applicants are hired into an applicant pool instead of 
for a specific position, placement strategies must also 
be developed. The best time for making an offer of 
employment varies and also needs to be determined locally. 



The interpretation of data is improved by using a single 
page summary document and training decision makers about 
how to process information. Errors frequently made in the 
interpretation process can be categorized as: leniency, 

halo effect, central tendency, logical, contrast, over- 
generalization, faulty weights, confounding^ and premature 
closure. Individuals making selection decisions need to 
realize they are making institutional decisions rather 
than individual ones. 



Improvement of selection processes depends 
ation of processes used in relation to the 
t eacher s selected . 



upon the evalu- 
success of the 
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Chapter 4 

THE EVALUATION OF TEACHERS 



Introduction Organizations may have as their primary thrust a motive 
for productivity (or profit) or a motive for survival, 

# Those organizations based on a profit motive have a 
high state of morale when productivity is high and 
when an informal atmosphere decreases distinctions 
of authority. Under such circumstances, individuals 
in the organization may welcome evaluation of their 
behavior and their output even though these evalua- 
tions are made by people in positions of authority. 

9 But where the primary thrust of the organization is 
survival, any aspect of the evaluation process may 
be suspect and its benefits questioned, regardless 
of who is involved. 

Some might use the above argument to conclude that emphasis 
on survival (or lack of a high profit motive) is the basis 
for teachers' resistance to evaluation; and such may be the 
case. However, there is some evidence (Rose, 1963) that 
teachers welcome evaluation if: 

0 the major focus is on improving rather than fault 
finding; 

• the information produced is meaningful to the teacher; 

0 the principal takes the necessary time to collect 
information that is adequate and to discuss it with 
the teacher. 

This research implies that there is a need to have agreement 
on purpose and procedures, and this can occur only when 
purposes and procedures are specific. Also communications 
must be honest as well as skillful, and disagreements must 
be handled in the open rather than avoided. 

NOTE 



Changes in teaahev or supervisor might be pre- 
cipitated as a result of evaluation, and both 
the evaluator and the teacher aj’e vulnerable 
when changes are possible. (position: research 

staff) 



Resistance 

by 

Teachers 
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Considerable resistance to discriminatory evaluation of 
teachers exists in spite of the advantage of such evaluation. 



CONCLUSION 



Removal of vesistanoe to evaluation depends on 
clear organizational goals ^ resources adequate 
for training evaluators and providing adequate 
time for them to perform the tasks required^ 
and clarity of the relationship of the organi- 
zational goals and the task of the evaluator, 
(practice: industry^ schools) 



Resistance 
by Other 
Forces 
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Changes in goals and procedures of evaluation are resisted 
by various forces, and teachers* organizations are one of 
the strong forces opposed to discriminating evaluations or 
evaluations which expand beyond the single prupose of im- 
provement of instruction. 

Although the research by Rose provides some indication of 
the nature of the resistance by teachers , what is the basis 
for the resistance by persons dong the evaluation? Inter- 
views with persons involved indicate a number of factors 
may be present • For example : 

• A general lack of certainty regarding criteria, 
measurement process, procedures for analysis and 
interpretation of data . 

• A resistance to placing oneself in position of 
manipulating or adversely affecting other people *s 
lives . 

0 A fear of precipitating an unpleasant reaction on 
the part of the person being evaluated. This re- 
action is then said to prevent a relationship that 
is conducive to helping the individual improve. 

• A lack of ability to cope with the weaknesses of the 
individual in terms of organizational needs and his 
ability to improve. This is sometimes linked with a 
failure to communicate to the individual the necessity 
of dealing with both the individual and the organi- 
zation * s problems , 

• A failure to see the relationship of evaluation of 
others to the purposes of the person doing the 
evaluation. 

• An inability to organize time in such a manner that 
adequate observations can be made. 
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Nature of In addition to the question of resistance to evaluation, a 

Evaluation number of other issues were encountered in discussions 
with members of organizations. For example, what is 
evaluation considered to be? If evaluation is a value 
judgment which occurs only following description, analysis, 
and interpretation, then it is discussed differently from 
evaluation viewed as a means of categorizing teachers for 
some obscure reason. How an evaluator defines evaluation 
is likely to determine whether he will be satisfying 
cognitive needs with knowledge or providing psychological 
support for eirotlonal reactions . 

• It appears that where evaluation is used for 
diagnostic purposes (i.e., where behavior is 
described and analyzed to help in determining the 
relationship between the process being used and 
the output of that process) the emphasis is 
justifiably on dealing cognitively with information. 

• However, if the emphasis is primarily on placing 
individuals in classifications or categories, as is 
the case when some cumulative rating is used, then 
one should be prepared to deal with emotional 
reactions . 

CONCLUSION 



Evaluation is a value judgment made late in a 
dynamic process; unless adequate description^ 
analysis^ and interpretation precede itj errors 
and poor results will occur, (position: 
research staff) 



Quality and Some propose that a major purpose of evaluation is to force 

Quantity of communication between the employee and the supervisor in 

Communication order that the employee can always know where he stands 
with the organization. Others indicate that quality of 
communication is important also, and that purposes of 
communication should go well beyond simply "letting the 
employee know where he stands." Continuous interaction 
between teacher and principal (supervisor) assists them 
to analyze information. This does not negate the use of 
formal written feedback at stipulated periods. To be most 
effective, the communication must be two-way, requiring 
that each person listen to the other. 

^ CONCLUSION 



Effective evaluation of teachers is dependent 
both on an adequate quality and quantity of 
communication hetb^een teachers and principals, 
(practice: schools) 
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Models of 
Evaluation 



Process 

and 

Product 



Goal 

Setting 



Wliat should be tlie model on which evaluation is based? 
Sliould it be modeled after a marriage contract ("for 
better, for worse," i.e., a tenure system) or after a 
professional baseball contract (no results, no renewal)? 
Would evaluation decisions and processes be any different 
if the teacher and principal had to agree on a contract 
annually? 

SUGGESTION 



Models of evaluation function within the con- 
straints of conditions of employment; more 
flexible models may develop if these con- 
straints are modified. (practice: industry) 



Some systems examine only the outcomes of teaching , 
ignoring what processes have been used. Other systems 
emphasize the classroom procedure, ignoring what has been 
learned by pupils. A self-correcting system should be 
designed which will identify errors and make changes in 
procedure before harmful effects occur. 

NOTE 



The purpose of exanining outcomes of teaching 
is to determine whether goals have been met; 
the purpose of examining procedures is to 
determine whether a specified plan is being 
followed; the purpose of comparing outcomes 
and procedures is to determine whether the 
procedures should be modified. (practice: 
indus try^ schoo Is ) 



An interview with the employee at the end of the first six 
months of employment may be used to set goals for the 
following year. This type of activity helps to coordinate 
the functions of selection and evaluation by integrating 
the personal plan for development with the evaluation 
procedure and also helps to express the organization’s 
philosophy. Then the evaluator can help establish a 
training program for the individual, rather than simply 
correct faulty behavior. In this way the evaluation is 
used in a positive manner to plan and direct the 
individual ’ s growth. 

NOTE 



Coordination of the personal and professional 
goats of a teaaher leads to improved morale 
and productivity . (practice: schools) 



Both the educational organization and individuals within 
it are being asked by the general public to be ’’accountable” 
for their actions and their products. 

NOTE 



AQQOuntability consists of providing evidence 
regarding the degree of accomplishment of pre- 
specified goals and objectives . Too often^ if 
a child does not learn^ the childy society^ or 
some factor other than the teacher is blamed. 
Teachers must dev^ lop a sense of responsibility 
for providing this evidence as Well as 
accomplishing goals, (position: research 

staff) 



Satisfaction with ’’accountability” probably is related to: 

• the degree of specificity of the goals 
o agreement on the goals 

• the perceived adequacy of the evidence provided 

A direct result of a desire for accountability is an 
increase in the number of states requiring annual 
evaluation of all (not just probationary) teachers. 



Purposes of Evaluation 

Although many people speak as though evaluation has a 
single purpose, it is apparent that there are multiple 
purposes and that sometimes these purposes are in conflict 
with each other. The difference between the schools’ need 
to satisfy the public’s demand for accountability and the 
desire of an individual teacher to improve the way he 
teaches a particular aspect of' one* subject suggests the 
wide extremes of purpose. The first consideration in 
establishing a program of evaluation, therefore, is a 
definition of purpose . 

The evaluation of teachers may serve many purposes: 

• to improve teaching 

• to reward superior performance 

a to supply information for modifying assignments 
a to protect either the individual or the organization 
a to validate the selection process 
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Improvement 

of 

Instruction 



Rewarding 

Superior 

Performance 



Modification 
of Assign- 
ment 



• to provide the basis of planning for individual 
growth and development 

• to facilitate self-evaluation 

There is general agreement among educators that the most 
important purpose for evaluating teaching is the improve- 
ment of instruction ( Oregon Education , 1966; Heald and 
Moore, 1968: Wagoner and 0*Hanlon, 1967); however, this 
improvment may take several forms: e.g., supervisors can 

provide feedback regarding behavior to teachers ; physical 
environment and materials can be modified; self-evaluation 
can be used to improve diagnostic skills of teachers; or 
information can be gathered by other teachers and dis- 
cussed with the teacher. 

Since evaluation of instruction is required before 
systematic improvement can occur, it is necessary to 
establish a starting point from which to work. As Heald 
and Moore (1968:189) state, "The routes to a particular 
end vary according to the point of origin , and it should 
be one purpose of evaluation programs to establish these 
points." Through this knowledge of strengths and weak- 
nesses a teacher can improve his work. Usually, when a 
teacher views evaluation as a means to improve his 
instruction, he accepts it as a part of the teaching 
assignment . 

Another purpose for evaluation is to make it the basis for 
rewarding superior performance . However , this use of 
teacher evaluation tends to meet with considerable 
opposition from teachers ,despitethefactthatpeople 
outside of schools are asking why teachers should not be 
paid according to the excellence of their performance, 
e.g., how well pupils learn (Fishman, Oregon Education , 
1966). These increasing pressures from school boards and 
taxpayers for rewarding superior performance are in direct 
conflict with the vie\<npoint of the majority of teachers. 

Some authors, Anthony (1968), Simon and Boyer (1967) and 
Hows am (1963) , suggest the teachers ’ maj or obj ection 
toward evaluation for the purpose of rewarding superior 
performance is due to subjective nature of the evaluations. 
They suggest the use of objectively obtained measurements 
of specific behavior which have been related by research 
to the accomplishment of specific pupil outcomes. Teachers 
also resent being classified into general categories of 
excellence, since excellence is specific to a situation 
as well as a person. 

Still another purpose of evaluation is to gather informa- 
tion for the modification of teachers’ assignment', either 
by promotion, reduction or increase of load, or release 
(Heald and Moore, 1968). While these are necessary 
activities among all organizations, morale tends to suffer 
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when evaluation emphasizes elimination of the weak and 
ineffective; consequently, this negative emphasis needs 
to be avoided. 

CONCLUSION 



The general contention is that better staff 
morale and a better instructional program will 
result from adequate and creative supervision 
and orderly dismissal procedures for incompetent 
teachers. (viewpoint: Eastmond^ 1959) 



Protection of 
Individual 
and Organiza- 
tion 



When the purposes of evaluation are considered from a legal 
standpoint, protection of individuals as well as the school 
organization becomes important. Responsibility for the 
operation of school programs is delegated by the state to 
the local district and from there to the teacher; with this 
responsibility comes the expectation that goals will be 
accomplished. Society does not expect schools to be 
operated without the same continuous evaluation that 
occurs in other organizations — although, from a legal 
standpoint, school boards have the right to establish the 
kind of school system they want as long as it remains 
within constitutional limits (Howsam, 1963 ; Heald and 
Moore, 1968). The boards’ prerogatives include estab- 
lishing any form of evaluation it desires. Because 
evaluation of teaching within a district serves as the 
district’s protection when it is held accountable for the 
system it has established, evaluation is essential for 
legal reasons — if for no others. 



NOTE 



The emphasis on the legal aspects of teacher 
evaluation can be viewed negatively by 
teachers unless they realize that their own 
protection against unjust charges is also 
assured. (Smith and Tyler ^ 1942) 



Validation 
of the 
Selection 
Process 



Though little recognized and practiced, another reason for 
evaluating teachers is to validate and improve the selection 
process. Development of procedures that link information 
analyzed during the selection process with teaching per- 
formance should be a part of every district’s planning. 

The criteria used to select teachers should be consistent 
with those used to evaluate teachers. 
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SUGCHSTION 



Promote 

Individual 

Grox>7th 



Facilitate 

Self- 

Evaluation 



Adverse 
Effects of 
Evaluation 



tlieVi-: /‘o a need for all who arc involved 
in avalualion to understand the 

relationships of on^aluation to selection pro- 
eesseSy school districts should consider 
involving supervisors in both selection and 
evaluation as one means to help them under- 
stand how the th)o ai'*c related. (practice: 
schools) 



Where principals are included in the selction process, as 
they are in many of the schools, it is expected that they 
are looking for the same qualities that they will later 
use for evaluation . As expressed by one personnel 
director, "They are much more careful who they select when 
they think the teacher may be placed in their own building. 
Also the principal may select according to his own parti- 
cular needs or program, such as ungraded classrooms or 
’’team” members with particular skills. 

Closely related to the improvement of instruction is the 
teacher’s o\^m purpose for evaluation, namely, to promote 
self-improvement . Since most evaluation procedures assume 
that each teacher is interested in improving his teaching, 
evaluation can become the basis of planning with the teache 
for individual growth and development (Howsam, 1963). To 
this end, new procedures begin with a goal-setting con- 
ference between the teacher and principal and means are 
then established whereby the teacher can acquire and 
diagnose information regarding his teaching. Although the 
criteria are individualized in this process and the 
diagnosis of behavior is done by the teacher, there is 
usually an attempt to mesh the individual goals with the 
institutional goals during the first conference . 

In addition, one of the functions of external evaluation 
is to facilitate self-evaluation. The value of self- 
evaluation is to allow continuous diagnosis of teaching , a 
very necessary component of good teaching . Since current 
ratios of principals (or other supervisors) to teachers 
do not allow for adequately frequent classroom observations 
and conferences, no teacher can afford to rely entirely on 
external evaluation. However, self-evaluation may lead to 
adverse effects if results are not checked by external 
observation. 

In addition to the potential benefits of evaluation, there 
are possible adverse effects as well. 

a There is the natural strain in human relations that 
results when one person evaluates another 
(Gruenfeld, 1966) . 
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o A conflict arises when both the function of 

accountability and assistance are expected from the 
same evaluation. It may be that a multiplicity of 
purposes is difficult to accomplish by a single 
evaluation procedure (Howsam, 1963). See also 
Performance Appraisal Interviews, page 174. 

o There is the tendency for the teacher to be shaped 
by a rating scale (or other measurement device) 
regardless of whether or not the scale validly 
measures good teaching. He may conform even though 
the measurement does not include necessary behaviors 
or in-cludes behaviors not pertinent to the work. 
Especially under situations of merit pay, states 
Link (1969), ”A rating scale becomes a shaping device 
no matter how supportive the supervisor, the principal, 
or the system. " 

Wagoner and O^Hanlon (1967) found that the extent of the 
benefit a teacher receives fro-n evaluation is related to 
his attitude toward evaluation, i.e., a teacher with an 
unfavorable attitude oenefits less than one who views 
evaluation positively. This implies that if teachers view 
evaluation as the attempt to eliminate the ineffective 
rather than to help all teachers toward better and better 
performance, adverse effects could outweigh the benefits 
of evaluation ( Oregon Education , 1966). 



Generally, a criterion is defined as a standard of level of 
attainment against which comparisons may be made. For 
purposes of selection , a criterion is that behavior (i.e., 
teacher performance) or result of behavior, (i.e., learner 
outcomes) , which one is trying to predict . In evaluation 
of personnel , a criterion also pertains to standards dealing 
with behavior and results-of-behavior ; however, rather than 
being something which one is trying to predict, it is used 
to assess the attainment of some process or outcome 
objective. 

Although the problems associated with criterion development 
for selection and evaluation purposes are similar, there 
are certain aspects of the criterion problem concerning 
teacher evaluation which evaluators should be concerned 
with. As a result, the purposes of this section are: (a) 

to discuss the nature of criteria dealing with teacher 
evaluation, and (b) to present certain ideas found in 
practice and the literature which should be considered in 
making decisions about the criteria to be employed. 



The Development of Criteria for the 
Evaluation of Teachers 
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Nature of Criteria 



CONCLUSION 



The nature of evo.luabion is similar to the 
nature of seleetioyi criteria in that criteria 
of teacner effectiveness vary from one job to 
another^ y and they tend to change over time, 
(viewpoint: Ghiselli^ 1956; and others) 



In the evaluation of teachers, most practitioners and re- 
searchers (Ware, 1964; Smith, 1967; Fishman, 1967; Ryans, 
1963, 1957; Barr, 1941; Schalock, 1967) agree that a 
teacher functions in a highly complicated setting , in 
which his performance is influenced by the interaction of 
his personal characteristics and various situational 
variables. Personal variables include such components as 
intellectual and affective structures , perceptual habits , 
age, and level of training; while the situational variables 
might include such components as the characteristics of the 
learners present, the materials being used, the goals of 
the institution, the instructional objectives, and the 
characteristics of the physical setting. 

Figure 4.1 presents representation of the interaction of 
the teacher's behavior and the situational components. The 
two-way arrows indicate the dynamic interaction between 
teacher behavior and the multitude of situational variables 




Figure 4.1. A Representation of the Interaction 

of Teacher Behavior and Various Situational 
Variables, (Modified from Schalock' s 1967 Model). 
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the other lines show the interrelationships of the situ- 
ational variables. As the diagram also shows, the 
situational variables provide the framework within which 
the teacher operates; however, since both teacher and 
environment interact with each other, there is a mediating 
effect on teacher performance. The implication of this 
diagram is that criteria determined for teacher performance 
should take into consideration the personal attributes the 
teacher brings to the job and the situational constraints 
placed on him by the job. 



Deciding on Criteria 



The initial step in teacher evali’.ation in a local school 
district is that of determining what is considered to be 
important in teaching. This first step is crucial to the 
evaluation process, since what is considered to be important 
becomes the basis for developing; 

• specific teacher behaviors and result s-of-behavior 
desired (identified as criteria) 

• ways to measure teacher behaviors and results (that 
is , criteria measurement) 

0 comparisons of measurements and desired outcomes 
(evaluation) . 

In determining what is considered to be important in 
teaching, i.e., the criteria, three questions must be 
answered -• 

• Who will decide on the criteria and their importance? 

• What procedure will be used to acquire information 
used in making this decision? 

e How will the data gathered be analyzed? 

Who will decide ? As mentioned in Chapter 3, criteria which 
are based on an individual's intuitive judgment are built 
on the weakest of foundrs'lions . Consequently , it was 
suggested that criteria decisions would be improved if 
based on the pooled judgments of experts. Ryans (1957:43) 
points out that the group of experts (jury of authorities) 
may be constituted in a number of ways: 

1. The jury may consist of the totality of the known 
group of authorities or experts (e.g., all of the 
principals and supervisors in the school district, 
all members of a teachers' professional organiza- 
tion, all college teachers of a specified subj ect 
matter, etc.). Of course, such a procedure usually 
is not feasible unless the totality of experts is 
relatively small. 





2. The jury may consist of a random sample from the 
roster or memebership list of a known group of 

uthorities . 

3. The jury may consist of a purposive sample drawn 

from the totality of authorities as defined. 

4. The jury may consist of a sample of individuals 

who have been specially trained to make authori- 

tative judgments regarding the criterion (e.g., 
job analysts, trained observers, etc. ) . 

In education, method 3 probably is most often employed; 
however, Ryans suggests that it is the weakest of the four 
techniques. He also warns that methods 1, 2, and 4 do '’not 
necessarily insure valid criterion description, but they 
represent distinct improvements” (1957:43). 

CAUTION 



Inadequate statements desevibing what is 
important in teacher behavior and results of 
behavior often result from using biased 
judges. (viewpoint: Ryans ^ 1957:43) 



A jury or group of experts in a school district, selected 
for purposes of criterion determination, also may include 
combinations of teachers, principals, supervisors, students, 
board members, and parents working together. Such a co- 
operative effort often has a positive effect on morale. 

CONCLUSION 



Involving teaoher^s as well as other members of 
the educational community in the development 
of criteria may help establish more accurately 
defined criteria and may improve the morale of 
the professional staff. (practice: schools) 



Where school districts require the establishment of goals 
for individuals, decisions regarding who will decide on 
what these goals will be are made by: (a) the individual 

teacher, (b) the principal or the supervisor, (c) the 
individual and the principal cooperatively, or (d) a 
committee composed of peers and/or administrators (the 
latter possibility might also include the individual 
teacher) . Also , in establishing the broad goals for 
individuals, board members, students, and parents may be 
consulted . 
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CONCLUSION 



Determining 

Criteria 



Regardless of how the goals are established ^ 
output and procedural goals are more likely 
to he understood and attained when they are 
cooperatively developed by the teacher a77d 
principal (or supervisor) and are written in 
discriminating behavior terms ^ (practice: 
schoo Is^ indus try ) 



What procedures to use in criteria determination ? Numerous 
procedures have been employed for acquiring information 
from people in order to determine what is considered to be 
important in teaching (Ryans, 1957; Ghiselli, 1955; and 
others). Ryans (1957:44) has listed six possible techniques 
which have been used for this purpose, 

• Free-response — statements of what is important and the 
degree of importance, based upon the general impressions 
held by various members of the educational community. 

• Check-list response — individuals indicate what is 
important and the degree of importance on a previously 
compiled list of desired behaviors and outcomes, 

• Position analysis — detailed systematic description of 
what is important for success and the degree of 
importance by individuals trained in carrying out such 
an analysis. 

• Critical incidents description — detailed descriptions 

of actual incidents and behavior that have been 
observed by experts to be "critical" in learner growth 
and development . Note : this technique primarily 

deals with teacher behaviors as opposed to learner 
outcomes . 

• Time sampling — detailed tabulation of teacher 
behaviors based upon systematic observation and 
recording, with special attention to the conduct of 
observations during representative samples in time. 

• Psycho-physical methods — members of the jury determine 
what is important and the degree of importance using 
such methods as ranking, paired comparisons, etc. 

In education, the first two techniques probably have been 
used more frequently than the others. However, from an 
objective and rational viewpoint, the first two methods 
are the weakest. In view of the nature of criteria for 
teacher evaluation, probably the method which holds the 
most promise is the "position analysis" technique. 

Evidence of the worth of this technique in improving the 
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selection process has been provided by Palmer (1970) in 
his research using a completed ^’position analysis outline” 
(PAO) for improved selection decisions. (See page 36 of 
this report for identifying the composition of the PAO.) 

How will the experts* responses be analyzed ? Once experts 
respond to the criteria dimensions they think important, a 
final judgment must be made concerning what criteria will 
be employed and how the criteria for evaluation will be 
operationally defined. It is recognized that regardless 
of whether an individual or a group makes these decisions, 
a value system will be involved; however, to make such 
judgments without a broader perspective of other’s views 
promotes a biased, unsystematic, and subjective approach 
which may tarnish the criteria descriptions. 

CONCLUSION 



VJhen deoisioTiS regarding the development of 
criteria are based upon empirically supported 
and rational considerations ^ relevance and 
usability are more likely to be ensured, 
(position: Ryans^ 1957) 



A number of authors (Guion, 1961; Ryans, 1957; and Brogden 
and Taylor, 1950) strongly recommend that a systematic and 
comprehensive approach be used to develop criteria. Such 
an approach would be designed to provide a rational analysis 
of the relevancy of the possible criteria; it would hypo- 
thesize descriptions of the criterion elements and use 
various statistical techniques (e.g., factor analysis) to 
identify significant operational behaviors pertinent to the 
attainment of the instructional obj ective (Ryans , 1957) . 

The significance of employing a more obj ective approach is 
that evaluation of teacher behavior and lea7:*ner outcomes 
can meet with success only to the degree that criterion 
j udgment is based on rel.iable information regarding the 
essential attributes and behaviors involved in teaching. 
Consequently, such information must be based on carefully 
defined and rigorously controlled evaluative research and 
not on "armchair” methods (Turner and Fattu, 1960). 



Some Important Teaching Behaviors 

This section has focused on the nature of criteria for 
assessing teacher effectiveness, and it has provided some 
suggestions as to the procedures for determining evaluative 
criteria. It has not been the object to specify what 
the important teaching behaviors are , because the behaviors 
and results-of-behavior deemed important for teacher 
effectiveness are specific to each situation. However, in 
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reviewing the literature, some behaviors have been iden*- 
tified which those responsible for criteria selection and 
development should be cognizant of. 

CONCLUSION 



Although the relationship of teacher behaviors 
to student outcomes is usually limited to a 
specific situation^ some behaviors have pre- 
cipitated desirable outcomes in more than one 
type of situation. (research: Flanders^ 1969) 



The following behaviors have precipitated desirable out- 
comes in more than one type of situation: 

• The teacher makes statements that use ideas and 
opinions previously expressed by pupils. This 
behavior appears to be a type of accepting or 
nurturant behavior. 

• The teacher uses a cognitive style which is flexible 
to the extent that he deals with two kinds of pupils 
differently or adjusts strategies from one time period 
to another. 

o The teacher uses a complex conceptual framework. This 
appears to be related to the idea that the diagnostic 
task is necessary to teaching, i.e., that assessments 
must be made in relation to goals and student errors, 
that choices must be made for remedial procedures. 
Immediate and long-range problem solution are facili- 
tated by this behavior. 

• The teacher provides a ”set** or "advance organizer." 
Students appear to process information more effec- 
tively when they are provided a framework within which 
to process the information. 

Procedures for developing criteria for teacher evaluation 
are moving from an inferential base to one that is much 
more descriptive. For example. Soar (1968) indicated that 
some of the newer procedures in developing criteria empha- 
size attempts to describe rather than evaluate. Criteria 
are developed that do not require value judgments on the 
part of the observer. Under these circumstances, the 
observer does not have to reach prior agreement about the 
nature of effective teaching. 
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Acquisition and Analysis of Data 
for Teacher Evaluation 



In practice, the activities of defining teacher performance 
criteria and identifying data collection and analysis pro- 
cedures parallel each other, i.e., their development is 
characterized more by interaction than by a sequential 
relationship. The criteria provide the focus of the 
evaluation by telling the evaluator and the evaluatee what 
behavior, objects, or events are important for determining 
performance success; the procedures of data acquisition 
specify how data will be gathered and the measuring instru- 
ments that will be used; and the techniques of data analysis , 
i.e., statistical methods and data processing, allow for the 
ordering of the data collected so that interpretations and 
conclusions may be made. 

SUGGESTION 



Evaluation procedures may he the same for all 
professi-oyial pevsonyiel regardless of level or 
position. This does not imply that the 
criteria will he the same; hut the procedures 
for establishing criteria the methods for 
collection and analysis of data^ and the use 
of data for decision purposes will he similar 
enough to incorporate into the same policy or 
strategy. (viewpoint: research staff) 



Planning 

Considera- 

tions 



Sources of 
Data 
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Planning for Data Acquisition 

Planning for data acquisition is one of the crucial steps 
that is often passed over lightly in evaluation , and con- 
sequently , many evaluation efforts fall short because of 
lack of it. Plans for data collection should take place, 
if not at the same time , iimnediately following the deter- 
mination of criteria for teacher evaluation. Included in 
the planning effort should be the following considerat ions : 

9 Where will the data come from? 

0 What form will the data be collected in? 

• \^at will be the sampling procedures? 

• Who will collect the data and what training will 
the collector need? 

One of the first considerations in planning for data 
acquisition is to identify the sources of the data. 
Typically these sources are identified early in the evalu- 
ation process at the time the criteria for evaluation are 





defined . However , it is important that the evaluator 
examine possible sources which may have been overlooked 
during the earlier stages so that he has information 
about the types of data that may be gathered and the types 
of measuring instruments which may be employed. Generally 
speaking, the sources of data include individuals who 
might observe classroom and out-of~classroom behavior: 

Classroom 

students 

teacher 

peers 

supervisors 
teacher aides 

Out-of-Classroom 

same as in classroom 
other para-professionals 
custodians 
parents 

other community members 

Because most data collected concerning teacher evaluation 
evolve from some "human activity," evaluators must be aware 
of the ethical standards involved in the collection and use 
of data in evaluation. As a result, the evaluator may wish 
to consult with people who acquire information as well as 
teachers to identify any limits in gathering data. This 
procedure may facilitate the collection of data by elimi- 
nating some of the resistance and negative feeling toward 
the data collector and the purposes of the evaluation 
effort. 

SUGGESTION 



Access to data may he facilitated if the 
evaluator y prior to data acquisition, con- 
sults with the sources of data to explain 
the purposes of the evaluation and to 
identify that information which is "off 
limits." (viewpoint: Bimbaum, 1970) 



A second consideration in planning for data collection con- 
cerns the form in which the data will be gathered, i.e., 
are the data in "raw" form or are the data in "reduced" 
form? Table 4.1 provides some examples of raw and reduced 
data which may be used for evaluating teaching behavior 
and the results of teacher behavior. 
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Table 4.1 



Examples of Raw and Reduced Data 
Gathered for Teacher Evaluation 





Raw Data 


Reduced Data 


Process 

Information 


audio or video 
recordings of 
teacher behavior 


supervisor's written 
comments about 
teacher ' s performance 


Outcome 

Information 


achievement test 
manuscripts 
answer sheets 


letter grades assigned 
by teacher 

students ’ grade point 
average 



Usually the purpose of the evaluation determines the form 
of the data desired, and the form of the data affects how 
it will be collected and analyzed. However , if the data 
are not readily available in the form desired, then adjust- 
ments in the collection and analysis procedures are 
necessary. 

Sampling The sampling procedures employed in evaluating teachers is 

Procedures another important consideration in planning for data 

acquisition . Sampling procedures have been designed to 
acquire only portions of the total amount of data available 
from the identified sources. In teacher evaluation, 
sampling techniques have been employed for two primary 
reasons: it is realistically impossible to collect and 

analyze all of the available data (e.g., one cannot hope to 
observe and analyze everything a teacher does), and the 
techniques allow the evaluator to distribute the information 
demands over the available time and data sources, in order 
not to overtax the endurance of any individual at any given 
point in time (Birnbaum, 1970). 

SUGGESTION 



Observation of teachers should he care fully 
spaaed over time to yield the best appraisal 
results. (viewpoint: Mitzel, 1967 j and 

others) 
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When employing observational techniques , systematic or 
non-systematic, as criterion measures, sampling is always 
a problem. It is important to note that when interaction 
analysis techniques are to be used, multiple observations 
need to be spaced carefully over time to adequately sample 
a teacher’s behavior (Mitzel, 1967, and Schalock and 
Beaird, 1968) . 

SUGGESTION 



\fhen samples of behavior are gathered by 
observational means^ teachers and students 
should be given some time to become 
acclimated to the presence of the observer, 
(viewpoint: Schalock^ 196?a) 



It is important that when observations are made either by 
a live observer or with the help of a video tape recording 
device, trial observations are conducted as the teacher and 
students become accustomed to having an observer in the 
class with them. Research findings are somewhat sparse as 
to the influence observation has on those being observed. 

Schalock (1967a) reports that some research (Thompson, 

1963; Paul, 1964) on mother-child interaction in the home 
suggests that as much as 5 - 7 hours of observation are 
needed before observer influence becomes constant. There 
is a good possibility that a shorter trial period would 
suffice in a classroom; however , until sufficient research 
has been conducted on this problem, data gathered using 
observational techniques should be interpreted with some 
caution. 

Training of A final consideration in planning for data acquisition 

the Data concerns the training of the data collector. If the data 

Collector to be collected are to be in raw form, little training of 

the collector is required. However, if data are to be 
reduced at the time of collection, then the collector must 
be capable of accurately recording and reducing the data 
for analysis and interpretation. 

For example, when studying the interpersonal communications 
between a teacher and students, an observer (data collector) 
must provide data which are valid and reliable. Therefore, 
the observer must undergo extensive training to memorize the 
definitions of the behavior categories used for classifying 
teacher and student messages; he must master the use of the 
behavior record form and the other recording procedures so 
that accurate judgments can be made; and he must demonstrate 
that his records are consistent from one observation to 
another. 
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SUGGESTION 



Individuals vesponsihle for acquiring ^^reduced^^ 
data should he provided extensive training in 
reduction procedures to ensure valid and 
r^eliable information. (viewpoint: research 

staff) 



Selection of Measuring Instruments 



In the evaluation of teachers, measurement techniques are 
central to the acquisition of data from the standpoint that 

• they sometimes influence the kind of information to 
be gathered, i.e., in considering what data are to be 
gathered, it is necessary to contemplate the type of 
measure that will be used; 

• they help order the available data; and 

• they help reduce the error of informal human 
observation . 

As Schalock (1969 , V-26) so aptly put it , 

In the absence of instruments for the extension of 
the senses, or for the control of conditions , human 
observations are liable to error. Instruments are 
a means for approximating more closely the property 
under observation . 

CONCLUSION 



Teacher evaluation is dependent upon measure- 
ment as a basis for information gathering^ 
because it is through measurement that i^e 
evaluator ascertains the quantity or quality 
of something. (viewpoint: Schalock^ 1968 ) 



Practical 
Criteria for 
Instrument 
Selection 



In selecting measures for evaluation, a major rule of thumb 
is ’^select the instrument which best fits your purpose,'* 
i.e., identify the measurement techniques and strategies 
which provide the data desired. Although this guideline 
is quite simple, too often choices are made on the basis 
of familiarity with or easy availability of instruments 
rather than because of their appropriateness. Obviously, 
such a situation is a case of the ’’tail wagging the dog.” 
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To help in making appropriate decisions regarding the 
kinds of measures to use or develop, Lyons (1970) has 
suggested four practical considerations or restraints: 



Other 

Considera- 

tions 



a Cost Factor - Priorities must be determined for the 
kinds of data needed and decisions made to allocate 
money among these priorities . 

o Time Factor - Soins measures take a great: deal of time 
to use and to develop properly and if not enough lead 
time is available, t>e use of such instruments will 
not be feasible. 

• Source Factor - It does no good to decide on a parti- 
cular instrument that would do the job, allocate 
appropriate resources , and then find out it is not 
possible to collect the data because no data source 
is available. 

• "Taboo^^ Factor - An otherwise satisfactory instrument 
can meet with resistance if it conflicts with local 
traditions or custom. 

The cautions discussed in Lyons' restraints reinforce the 
idea that planning for data acquisition and the instruments 
to be employed is essential in establishing teacher evalu- 
ation procedures. 

Besides the practical considerations listed above for 
choosing or developing an instrument for teacher evaluation, 
one should also consider the characteristics which indicate 
adequacy of any measuring instrument, viz., relevance, 
reliability, validity, fidelity, and ease of administration. 
A number of authors (Lyxons , 1970; Schalock, 1968; Kerlinger, 
1964; Thorndike and Hagen, 1962; Ryans, 1957; and others) 
describe those characteristics as follows: 

• Relevance — This quality is sometimes referred to as 
validity, i.e., the extent that the measure appears 
to be measuring that which it says it is. 

• Reliability — This quality concerns the consistency 
or reproducibility of the measure, i.e., the 
instrument continues to maintain its stability from 
one application to the next. 

• Validity — This quality pertains to the fact that the 
instrument measures that behavior, object, or event 
for which it was intended to measure. 

• Fidelity — This quality relates to the degree to 
which the response co the instrument parallels the 
true or actual performance (e.g., skill tests in 
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physical education have a greater degree of fidelity 
than do paper and pencil tests). 

• Ease of Administration — This quality involves the 
practicality of the instrument in the evaluation, 
i.e., its availability, scoring ease, etc. 

CONCLUSION 



Data acquired for teacher evaluation purposes 
way be analyzed and interpreted with a greater 
degree of confidence if in the evaluation pro- 
cess the following two questions can be 
answered positively : 

0 Hill the measuring instruments employed 
fit the purpose of the evaluation and 
will they do an adequate Job? 

0 Have the measuring instruments been 
implemented accurately? 

( viewpoint: research staff) 



Some Uses of Measuring Instruments 



Thus far, this section has focused on some approaches to 
planning for data acquisition and on some criteria for 
selecting appropriate criterion measures. The purpose of 
the remainder of this section is to present some of the 
techniques for gathering data for teacher evaluation. 

A variety of measuring instruments are available for 
obtaining data for purposes of teacher evaluation. To 
provide the reader with some information as to how these 
measures have been employed, the discussion has been 
organized into three broad categories ; 

• direct measurement of teacher behavior 

• indirect measurement of behavior 

• measurement of the results-of -teacher-behavior — i.e., 
learner outcomes. 
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Direct Measurement of Teacher Behavior 



In-classroom Direct measurement of teacher behavior refers to the 
Behavior assessment of a teacher’s performance as he attempts to 

influence learner development within some instructional 
setting. This form of assessment employs measures ranging 
from highly systematic measurement techniques (e.g., 
systematic observation using trained observers) to less 
systematic techniques (e.g., casual observation by un- 
trained observers). 



Systematic Observation Procedures . * Measures in this 
category are designed to minimize, as much as possible, 
Character- the influence of observer bian. Characteristics of these 

istics measurement techniques i^iclude: 

9 prior analysis of the criterion behaviors; 

• clearly defined behavior category dimensions; 

e use of an observational record for recording purposes; 

• establishment of inter-rater and intra-rater 
reliability ; and 

• intensive training of the observers in observational 
and recording techniques . 

^ CONCLUSION 



Systematic observational procedures are 
designed to minimize and control j as much as 
possible, the influence of observer bias in 
instrument application. (viewpoint: Medley 

and Mitzel., 2963; and others) 



Complexity Systematic observational technique:- generally are categorized 

of Systems as either single-factor systems or multi-factor systems 

(Medley and Mitzel, 1963; and Losenshine, 1970). That is, 
either they are intended to focus on one aspect of the 
teacher's behavior at a time (e.g., the number and type of 
questions transmitted during a given lesson), or they are 



*For an extensive disccs-sion of systematic observational 
procedures, see Medley and Mitzel 's chapter, "Measuring 
Classroom Behavior by Systematic Observation," in Gage's 
(ed.) Handbook of Research on Teac. ing (1963). For a 
review of observational devices, see Simon and Boyer, 

Mirrors of Behavior (1967 - 1970). (See Bibliography.) 
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intended to focus on two or more aspects (e.g., analyzing 
the verbal and nonverbal messages transmitted between 
teachers and pupils as well as the changes in the instruc- 
tional setting during a given lesson). 

NOTE 



Single- factor systematic observation systems are 
quite manageable by local school officials in 
measuring teacher behavior. (viewpoint: 
research staff) 



Because of the complexity of the multi-factor systems, 
local school officials may find single-factor systems (e.g., 
Flanders Interaction Analysis System) more manageable. 

The basis for this position does not rest on the ability 
of those in the local setting, but on the practical reality 
that the complexity of the multi-factor systems require 
intensive training and retraining of observers. Also, such 
systems may not provide a justifiable return for the 
expenses incurred . 

Histor'^ The rise of systematic observational procedures for 

studying teacher effectiveness dates back to a study by 
A. S. Barr reported in 1929, Characteristic Differences 
in Teaching Perf ortnance of Good and Poor Teachers of the 
Social Studies . 

Many of the studies employing these techniques, prior to 
the 1950*s, *'took the form of supervisors * check lists 
designed to rate teachers rather than to study teaching." 
(Kliebard, 1966 : 46) . The greatest use of systematic 
observation procedures has been within the past two 
decades, and most of these studies have been of a des- 
criptive nature rather than evaluative (see Medley and 
Mitzel, 1963; and Flanders, 1969). The major purpose 
of these studies has been to study the interaction 
between teacher and pupil by keeping a running record of 
selected behavioral events that occur within the class- 
room. 

Applications Flanders in his recent book. Analyzing Teaching Behavior , 
has illustrated the two major applications of interaction 
analysis in the study of teacher behavior (1970:3). 

• One application of these activities is to help an 
individual develop and control his teaching 
behavior . 
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Choosing 

Appropriate 

System 



Increase 
in Use 



• A second application is to discover through research 
how to explain the variations which occur in the 
chain of classroom events. These explanations are 
meant to focus on teaching behavior and its relation- 
ships to classroom interaction and educational 
outcomes. 

CONCLUSION 



The appropriateness of the systematic observa- 
tion system to be employed is directly related 
to the kind of info motion desired. (viewpoint: 
Schalockj 196?a) 



A basic question thac needs to be asked when system^^cic 
observation techniques are to be employed is , ’’Does the 
category system fit the purpose for which the data are 
being collected?" Since all observation systems have 
different strengths and weaknesses, it is important that 
one should do a thorough analysis of the kind of informa- 
tion desired before selecting a procedure to be used. 

That such an effort is required is exemplified in a recent 
study by Medley and Hill (1969) in which the\ compare the 
Flanders instrument with Medley and Mitzel’s OScAR 
(Observation Schedule and Record) instrument for analyzing 
teacher-learner interaction. They reported that Flanders^ 
technique was easier to use and was more sensitive to 
student behaviors, while OScAR focused more on instruc- 
tional and management behaviors. They concluded that both 
systems are good, but the user must decide which is best 
for what he is trying to measure. 

Schalock (1967a) suggests that if verbal behavior is the 
object of focus, then behavior should be recorded with 
audio tape and a system such as Flanders Interaction 
Analysis System employed. If both verbal and nonverbal 
behaviors are desired, then a multi-factor system such 
as the Teaching Research System for the Description of 
Teacher Behavior in the Classroom (Schalock, 1967b) can 
be used for directly observing a classroom or observing 
a videotape recording of a classroom interaction. 

Reviews of research findings (Medley and Mitzel, 1963; 
Soar, 1968; Flanders, 1969; and Rosenshine, 1970) indicate 
that studies using systematic observation procedures 
(especially interaction analysis) to describe teacher 
behavior are increasing. Also increasing is the develop- 
ment of new observation systems (Simon and Boyer, 1967, 
1970: and Boyd and DeVan.lt, 1966). However, the state of 



the art is still in a somewhat primitive stage. Con- 
tributing to this sicuatTon is the fact that it is 
difficult for prospective users to obtain good instruments 
from the original authors. Where the author has made it a 
point to disseminate the observation system he has developed, 
this problem has been alleviated. For example, Flanders’ 
dissemination of his interaction analysis technique through 
workshops, monographs, etc. has contributed to the more 
frequent use of his instrument. 

Adaptation A second contributing factor to the primitive state of the 
ii:teraction analysis art is the difficulty practitioners 
have in adapting a particular system to their particular 
needs. This problem has been identified in discussions 
with numerous school people, and it is one of the many 
areas in which training and dissemination workshops are 
needed . 

Some exceptions exist ; for example , Flanders ’ system has 
been adapted to the needs of its users to some degree 
(see Amidon and Hough, 1967). Another system that is 
currently addressing itself to the problem is a system 
developed by Parsons (1969), known as Guided Self-Analysis . 
Through much effort, Parsons has programmed the learning 
of his system so that teacher and supervisors can use only 
those parts of it v/hich fit their needs . 

SUGGESTION 



Training both supervisors and teachers in the 
use of an interaction analysis system will 
provide a common language for analyzing the 
teaching- learning process . (viewpoint: 

Flanders^ 1963^ and others) 



Research studies which have used interaction analysis 
techniques show that our understanding of the structure to 
measure desired outcomes and function of the verbal chain 
of events in the instructional setting has greatly improved 
(Flanders, 1970). In addition, the training of both 
supervisors and teachers in the use of interaction analysis 
has provided a common language for analyzing the teaching- 
learning process (Flanders, 1963; Hill, 1966; Thoreson, 
1963; Moskowitz, 1967; Hough and Ober, 1966; and Malone, 
1968). 

Non-systematic Observation Procedures . Direct measurement 
of teaching behavior also includes those non-systematic 
observational techniques which are either "analytical” or 
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“general’’ (Ryans, 1957:52). Traditionally, in evaluating 
teacher behavior via observational means these procedures 
have been employed in one way or another because they are 
easy to construe.'* and use. Generally, analytical pro- 
cedures Include renting scales and check lists whereas 
general procedures consist of broad or global assessments 
of teacher behavior. Although analytical techniques are 
somewhat more systematically structured than those in the 
general category, they are still prone to error. This is 
primarily due to the characteristics common to both 
analytical and general techniques — the observer , trained 
or otherwise, withholds his judgment until the end of the 
behavior sequence. This contrasts with interaction 
anaJysis techniques which usually consist of a continuous 
record of predetermined categorical sets. 

CONCLUSION 



The chief advantage of non- systematic 
observational techniques (e.g.^ rating scales) 
is that the observer is able to consider clues 
from a variety of sources before he makes his 
judgment. (viewpoint: Rosenshine^ 1970) 



Studies of teacher evaluation have been reported in which 
pupils, peers, and supervisors have assessed teacher 
behavior using various non-systematic observational 
techniques. In many cases results indicated discrepancies 
in the observer ratings. For example, Poppleton (1968) 
used a modified critical incident technique to identify 
discrepancies in the assessment of student teachers by 
supervisors and cooperating teachers. Her results show 
that the ratings of the cooperating teachers were normally 
distributed, but the grades assigned by the supervisors 
were negatively skewed in the B grade category. “This,” 
she concludes, “no doubt arises from the recognition of 
the seriousness of failure by those responsible for 
awarding the certificate.” 

What can be done to guard against the problems rising from 
subj ect judgments and rater discrepancies? The following 
suggestions are made to answer this question: 
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SUGGESTION 





To improve the acouraQij of using non- systematic 
observatioj'ial techniques the following might he 
considered: 

0 Clearly define the focus of the evaluation; 

0 Develop specific^ low inference items'^; 

0 Use a common record form; and 

0 Provide adequate training for the 
observers. 

( viewpoint: research staff) 



Self-evaluation of behavior . A third means to direct 
measurement of teacher behavior is having the teacher 
evaluate his own behavior. Self-evaluation of behavior 
differs from other direct measurement procedures in the 
following manners: 

• There is no need for involving an external 
observer in the measurement urocess. 

• Behavioral criteria are determined by the 
teacher rather than by some external sourc^j. 

The advantage of this approach is based on the idea that 
evaluation should be an internal process rather than an 
external one (Rogers, 1954). 

CONCLUSION 



The chief advantage of the self-evaluation 
approach is that the teacher has the opportunity 
for self-improvement without an external threat. 
The chief disadvantage is that the approach does 
not readily relate to an outside criterion, 
(viewpoint: Ryans^ 1957; and others) 



^Following his recent review of the use of rating scales 
for instructional evaluation, Rosenshine (1970) reports 
that specificity of items enhances the interrater relia- 
bility to the point where such measures are comparable to 
systematic categorical observation systems . 
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Problems 



Out-of- 

Classroom 

Behavior 



The self-evaluation approach for teacher pre-service and 
in-service training has been subjected to a number of 
studies, some of which have permitted total self-analysis 
by the teacher (Jensen, 1968); others have included an 
individual or group for critiquing purposes (Thoreson, 
1963; Bush and Allen, 1964). Two other disadvantages, 
other than the one menf^oned above, have been found: 
teachers are not able to analyze specfic aspe'^ts of their 
behavior because they are v/ithout a conceptual frame or 
systematic observation system, and teachers lack technical 
competence for operating the audio or video equipment 
necessary for capturing their behavior for analysis. 

SUGGESTION 



Before implementing the self-evaluation 
approach for improving instruction: 

# Teachers should be provided with a frame- 
work (an observational system) for 
analyzing and interpreting their own 
behavior . ^ 

0 Teachers should be provided with the 
technical competence needed, for 
operating the various new media used for 
recording their behavior. 

(viewpoint: research staff) 



Indirect Measurement of Teacher Behavior . Indirect 
measurement of teacher behavior deals with assessment of 
out-of-classroom teacher behavior such as activity in 
' 'rganizations , extraclass activities, contribution to 
curriculum development, interactions with parents, hobbies, 
and education. Although indirect measurement sometimes is 
not incorporated in any formal evaluation of teacher 
behaviors, such information is ample in quantity, often 



*The notion for providing a teacher with some framework for 
evaluating his own behavior stems from the rationale that a 
personas predisposition affects his perception, i.e., what 
is in a person’s ’’head” gives him the set for what he sees. 
A good analogous situation is the biologist who, without a 
taxonomy for categorizing what he observes through the 
microscope , makes only gross observation . 
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pertinent, and relatively easy to obtain. Weaknesses of 
such procedures include the problem of limited sampling 
of the behaviors being measured and scoring difficulty 
(Schalock, 1969). 

CONCLUSION 

Advantages of measuring out-of-olassroom 
teacher behavior are: 

0 Information is ample ^ often pertinent 

0 Data are easy to obtain 

Disadvantages are: 

0 Occurrence of limited sampling 

0 Occurrence of scoring problems 

(viewpoint: Schalock^ 1969) 



An example of indirect measurement of teacher performance 
is a study conducted by Popham and Standlee (1958) , which 
was designed to explore the relationship between teachers’ 
uut-of-school activities and their professional performance. 
Principals’ ratings and the Minnesota Teacher Attitude 
Inventory (MTAI) were employed as measures. Four con- 
clusions were stated in the study: (a) professional 

performance does not relate to the whole range of out-of- 
school activities, rather only to particular activities; 

(b) measures on the MTAI are related to the professional 
out-of- school activities of the teacher ; (c) performance , 
as measured by the principal’s ratings, is related to non- 
professional activities; and (d) the MTAI and principals’ 
ratings assess different dimensions of the teacher’s 
professional performance (p. 29). 

Measurement of Pupil Outcome . Many educators agree that 
the most satisfactory criterion measure is the product of 
performance; the emphasis is on the result or outcome of 
instruction rather than the process of instruction. The 
major reason for preferring pupil outcomes as the measure 
of teacher effectiveness is that the goal of teaching is 
learner development, and therefore the teacher should be 
accountable by providing evidence that learning has 
occurred . 
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Measures 



Pupil outcomes include how pupils think, perform, and feel. 
Consequently , the traditional measures used for gathering 
data for these preferred criteria include: 



• knowledge and ability measures — tests designed to 
measure what a person knows 

• skill performance measures — tests designed to 
measure x^7hat a person can do 

• att i tudinal measures — measures designed to assess 
a person’s feelings or desires 

• interest measures . 



Numerous writers have discussed in great detail the con- 
struction and use of instruments for measuring pupil 
outcomes . As a result , no discussion of these specific 
measures will be provided in this paper.* * 



*For those who are interested in a more intensive dis- 
cussion about the selection and construction of specific 
instruments , see the following sources : 

• Buros, Oscar Kristen (ed.) The Sixth Mental Measure- 
ments Yearbook . Highland Park, N. J.: Gryphon Press, 

1965. 

• Cronbach, Lee J. Essentials of Psychological Testing . 

New York: Harper Brothers, 1960. 

• Guilford, J. P, Psychometric Methods , New York: 
McGraw-Hi.il, 1954. Chapter 2 Rating Scales, pp. 262- 
301; Ch. 15, Test Development, pp. 414-469. 

• Kerlinger, Fred N. Foundations of Behavioral Research : 
Educational and Psychological Inquiry . New York: 

Holt, Rinehart and Winston, Inc. , 1964. Part 7, 

Methods of Observation and Data Collection, pp. 463-599. 

• Schalock, H. E. Measurement . In CORD: National 

Research Training Manual . 2nd Edition. Jack Crawford 
(ed.) Monmouth, Oregon: Teaching Research Division, 

Oregon State System of Higher Education. Chapter V, 

85 p. 1968. 

» Super, Donald E. and John 0. Crites. Appraising 

Vocational Fitness . New York: Harper and Row, 1962. 

• Webb, E. J. , D. T. Campbell, R. D. Schwartz, and L. 

Sechrest. Unobtrusive Measures: Nonreactive Research 

in the Social Services . Chicago: Rand McNally and Co., 

1966, 225 p. 
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Problems Altliough a product criterion measure is preferred, its 

use presents a number of problems in teacher evaluation. 
As Ryans (1957:55) states: 

The chief disadvantage in the use of products as 
criterion measures is tlie difficulty of adequately 
controlling external factors in order to provide 
reasonable assurance that the hypothesized pro-* 
duct is truly a product of the criterion behavior 
rather than that of a wide range of uncontrolled 
conditions occurring prior to and concurrently 
witli the criterion behavior . 

CAUTION 



As the complexity of the situation increases 
and control declines^ the ability to attribute 
pupil learning to a specific teacher ^s behavior 
diminishes. (viewpoint: Ryans ^ 1957) 



The control question discussed by Ryans constitutes one of 
the major problems in evaluating teacher effectiveness. 
Whereas experimental research uses the laboratory to handle 
such a problem, evaluation in the field has had difficulty 
in dealing with it adequately. As a result, evaluative 
study of pupil outcomes is often confounded and contaminated 
by the effects of various uncontrolled factors . 

Research The study by Turner, ^ , (1963) represents an example 

Example of coping with the product criterion problem. The purpose 

of the study was to examine the effect of problem-solving 
ability of teachers, as measured by the ’’Mathematics 
Teaching Tasks,” on pupil achievement and supervisory 
ratings of the skills of teachers in teaching the unit. 
Results from the study indicate that pupils under two 
successive high- scoring mathematics instructor s , perform 
significantly better than those under pairs of low-scoring 
teachers. 

SUGGESTION 



Wl'ien using pupil aakievement as a ovitevion 
measure, aonsideration must he given to the 
suggestion of teachers influencing the pupil's 
achievement. (research: Turner, et al, 1962) 
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Besides focusing on learner outcomes in terms of pupil 
gain scores , assessment of pupil behavior in the class- 
room, which is assumed to be influenced by teacher 
behavior, may be conducted (Ryans, 1957 ; and others). 
Generally, this form of evaluation has been found most 
^'ften in situations where skill performance on the part 
of the learners is the primary object of instruction 
(e.g., drama, speech, and physical education classes); 
however, in view of the increasing use of systematic 
observational procedures for assessing teacher verbal 
''"ehavior, it appears that in the not too distant future 
observational techniques will be modified for the purpose 
of describing the pattern and level of pupil verbal and 
nonverbal behavior in classes intended for cognitive and 
attitudinal development. 

Some Procedures for Data Acquisition in School Districts . 
Discussions with people in various organizations leaves 
the impression that a variety of evaluation procedures 
are being used. "Management by objectives" procedures 
have been in use for several years in business organiza- 
tions and are beginning to be discussed seriously in 
school systems; self-evaluation procedures are seen by 
many as being the most satisfactory way to improve instruc- 
tion; and peer evaluation holds promise for others to 
satisfy requests for accountability . 

One view which may be pertinent to many of these procedures 
was expressed by a supervisor who operates within a 
"management by obj ectives" framework: "The individuals 

who indicate job satisfaction are those whose supervisor 
communicates with them regarding their performance." 

The following ideas seem pertinent to acquisition and 
analysis of information in school systems : 

• Teachers and administrators benefit from a school 
board policy which identifies the purposes and general 
procedures to be used in evaluation of personnel ; in 
addition, a benefit is derived when the school board 
identifies what types of data on teachers it desires 
in order to establish policy and provide information 
to the public. 

• If an evaluation form is being used with a group of 
teachers, all new teachers should examine the form 
early in the year (and perhaps complete some of the 
information) in order to become thoroughly acquainted 
with the criteria for evaluation. 
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• Vice principals and department chairmen are involved 
in collection of information, but their role in the 
total evaluation process may be different from that 
of the principal . 

• 1-Jhere student evaluation is used on a formal basis, 
the student reactions are usually anonymous and 
acquired at the option of the teacher . 

• One of the limitations of peer evaluation is the 
hesitancy of peers to make critical comments . 



Purposes of 
the Post- 
Observation 
Conference 



Post-Observation Processes 

I^en should decisions be made regarding needed supervisory 
treatments , change of assignment , promotion, or release 
from emp loymen t ? 

• Should the supervisor enter the t-*observation 
conference with decisions clearly in mind? 

• Should he make joint decisions with the employee 
during the process of the conference? 

9 Or should the conference be considered part of the 
information-gathering process and decisions be 
delayed until after the conference? 

The answer to these questions are not easily found in 
research on teacher evaluation. Noland and Moylan (1967: 
vi) indicate that a wide variety of practices occur in 
business . 



Some firms use either an informal or a standardized 
rating sheet, but do not make the results known to 
the employee . Other organizations use a standard 
evaluation procedure and rating form and evaluate 
the employee in an interview at which he is present. 
Still others see the employee only after the evalua- 
tion has been completed . 

In both business and schools some kind of meeting between 
supervisor and employee often follows observations of the 
subordinates^ work. Usually this meeting is referred to 
as an "evaluation conference" by educators and as a 
"performance appraisal interview" by business personnel. 
Essentially, both are used to fulfill the same purposes. 

In business the diversity of these purposes has greatly 
increased since the appraisal interview has been seen as 
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a means to develop subordinates as well as evaluate tliein 
(Solem, 1960). 

NO’id 



porfcrrnance appri\.sal interview can involve 
such di^'- irse functions as the evaluation of 
porf'orwance, motivation of the subordinate, 
warning, praising, developing , treating the 
suhord.i. ,ite as an individual, recommending 
future courses of action, and the differential 
granting or withholding of an entire system of 
rewards and pu> . <.shments . (research: Solem, 

1960) 



Effectiveness 

of 

Performance 

Appraisal 

Interviews 



Research Ci^mducted uy Meyer and Walker (1961); and Meyer, 
Kay and French^ (1965) attempted to assess the effective- 
ness of performance appraisal interviews. They reached 
four conclusions : 

Criticism has a negative effect on the achievement 
of goals because it builds defensiveness . 



« Praise has little effect one way or the other on 
the achievement of goals . 

0 Mutual goal-setting improves performance. 

0 Coactiing should be day-to-day, not a once-a-year 
activity. 

Rose (1963) reached the same conclusion that the manner in 
which feedback is provided to tecchers can halt or enhance 
growth and improvement. Treating each teacher as an 
individual, according to how each responds to supervisory 
help can be accomplished through the appraisal interview 
or cc if erence . 



CAUTION 



±he teaohev no-^e readily accepts decisions based 
on observations and conferences if the mayor 
focus is on inrproving rather than fault finding, 
(research: industry j schools) 
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Rigid feedback procedures need to be carefully evaluated. 
The number of improvements tliat can be accomplished at 
one time is limited , and teacliers must be spared the over- 
whelming feeling of failure accompanying a long list of 
’’needed improvements” (Noland and Moylan, 1967). Wliil.e 
even the successful teaclier has failure experiences , he 
should not be led to believe that he has so many as to 
lose his sense of personal worth or emotional stability. 

On the other hand, the tendency of supervisors to be 
overly generous with praise causes appraisals to have 
little value as a basis for making decisions (Lopez, 1965; 
French , J , , 1966) . 

Prior Goal Since a variety of decj.sions may be made on the basis of 

Setting the evaluation of teachers, it is essential that all who 

are involved in the system understand the purposes and 
procedures as well as the roles of the various partic-- 
pants. For example, a **ierit promotion board might review 
evaluation data using a different procedure from that of 
the principal or supervisor assisting the teacher in 
instructional methods. Likewise, a group decision might 
be made by an evaluation committee responsible for 
reviewing all probationary teachers for retention. Unless 
differing roles and procedures are understood by all 
involved , misunderstanding and low morale can result . 

SUGGESTION 



Plans regarding supervisorij treatments, 
asstanments , promotion and release should 
become part of a goal-setting aotivity in 
order for all participants to understand 
the bases for evaluation. (position: 
research staff 



Many authors refer to the importance of prior (and mutual) 
goal setting. For example, Rudfern (1963) prefers that 
the evaluation of teachers ir. an ’’umpiring" sense be dis- 
continued in favor of a "coaching" relationship . The 
joint establishment of performance goals and standards 
before and during the school year reduces the need for 
the traditional annual rating decision . Rather , through 
con trast:^ng his achievemen t with predetermined standards , 
the teacher knows how he is doing at all times. Just as 
Meyer (1961, 1965) cited the advantages of mutual goal 
setting, Odiorne (1965) emphasized the need for a standard 
of performance which has been made clear to provide 
motivation toward specific goals. Except when related to 
the next set of goals, making judgments after performance 
has occurred has little value (McNeil, 1967). 
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Change of 

Assignment 

Decisions 



Designing 

In-service 



Personnel 

Files 



A decision to change the assignment of a teacher is 
sometimes made on the basis of evaluation of performance. 

Such a change is j ustif ied on the assumption that teachers 
behave differently in different situations. For example, 
a given set of teacher behaviors will affect one group of 
students differently than another group; therefore, a 
teacher exhibits more effective behavior in one situation 
than in another. This assumption appears to be justified 
by the research literature (Flanders, 1969; Mitzel, 1960; 
Ellena, Stevenson, and Webb, 1961). 

Most decisions made in the evali'>ri'.n process are based on 
information regarding the individual teacher. When the 
data from individual teachers are cumulated and analyzed, 
the information may be useful for making decisions concerned 
with the design of in-service courses for groups of teachers. 
Such courses can be designed around weaknesses that are 
reported through the evaluation process . 

One of the most significant issues in record keeping for 
evaluation purposes is policy regarding personnel folders. 
Policy generally takes one of two forms: either the 

personnel file is open to the teacher at his request, or 
it is open to administrative personnel only. 

CONCLUSION 



Open personnel files and reports to the Boards 
of Education tend to reduce anxiety and rumor 
among those being evaluated, (practice: 
education) 



When the file is closed to the teacher, it is assumed that 
the file contains confidential recommendations obtained 
for promotions. Under the circumstances, some systems 
provide copies of all evaluation reports ' the teacher ; 
some systems require four copies so that Lne principal , 
the personnel director, and the assistant superintendent 
(or curriculum personnel) receive copies also. One 
central office person is then responsible for reviewing 
each evaluation report to suggest actions that might be 
taken for improving individual teachers or to recommend 
changes in the system. When the official record system 
is open to the teacher, principals sometimes keep a 
ptirsonal file which is not officially a part of ' e 
district ’ s record sys.tem. * 



^^This tends to subvert the intention of open files, but 
nevertheless is practiced . 
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Assessment 


of the 


Evaluation Process 




Need 


for 


Too often, after a 


system 


has been 


developed for 


the evalu 


Asse 


ssment 


ation of teachers. 


school 


s neglect 


to develop a 


systematic 



of Process approach to judge the effectiveness of that system. If the 

teacher evaluation process is not periodically analyzed for 
problems , and if concern for assessment of teacher evalu- 
ation only occurs when any one particular teacher seems in 
difficulty and is in jeopardy of losing his position because 
of lack of competence or effectiveness, it becomes very 
difficult to discover who really is at fault: the individual 

teacher , the system for teacher evaluation , some aspect of 
implementing the evaluation system, or a combination of 
these elements . 

IVhen a school district continuously monitors the teacher 
evaluation process, they have constant sources of feedback. 
This allows for anticipation of problems and, as in the 
case of the ineffective teacher, may point to modification 
in supervision before problems reach a point where alterna- 
tives are limited to forced resignation or cancellation of 
contract . 

Problems Some evaluation problems may not derive from the system 

itself nor the intended procedures but may be due to prob- 
lems of implementation . For example, if adequate time is 
not spent in observation and providing feedback to teachers, 
the goal of instruction improvement may not be served. On 
the other hand, a thorough examination of goals may indicate 
that they are unreasonable for a particular teacher or group 
of pupils. Other sources of problems may be procedural . 

NOTE 



AnaZysis of difficulties necessitates 
of the realism of goali the adequacy 
implementation^ and the effectiveness 
ing procedure . (practice : schoo Is) 



examination 
of their 
of tf ^ teach- 



At the heart of assessment of the evaluation process is the 
coordination of information sources which supply data for 
analysis of difficulties. If one is particularly concerned 
with the problem of retention of teachers, personnel pro- 
cedures might include the early period of employment as an 
extension of the selection process. This emphasizes the 
need for coordinating the evaluation process with other 
personnel procedures , and for developing a record system 
which facilitates this coordination. 
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NOTE 



/i conipreheyisive personnel record system will 
facilitate tfie coordination of selection ^ 
evaluation ^ and supervision-of-assigmient 
func t ions . ( practice : indus try ) 



Sources of Usually, the personnel record system contains the data 

Information acquired during the selection process and the formal 

evaluative comments from principals or supervisors. An 
additional source of information which is useful and easy 
to obtain is the exit interview . When teachers leave a 
school system , information can be sought regard ing : 

0 why they are leaving the system 

9 teacher-perceived problems in supervision procedure 
(which includes evaluation practices) 

o teacher -perceived problems in selection and placement 
procedures . 

Because it has been noted that one of the problems involved 
with evaluation of teachers is due to the lack of training 
of those who do the evaluation, this is a problem to antici- 
pate . 

SUGGESTION 



'Puh'lic sohool systems often treat the training 
of those who evaluate teaoherc rather casually. 
However^ training of personnel involved in 
teacher evaluation is likely to increase 
Validity, reliability, disorimination, and 
feelings of certainty regarding decisions, 
(practice: industry) 



Many times a vievjpoint is expressed that school administra- 
tors are professional people, they know their job and what 
they desire in teachers, and they know how to determine when 
good teaching is occurring. This view is expressed in spite 
of research which indicates that the reliability and validity 
of administrators' judgments is generally low when compared 
with student gain (Ellena, Stevenson, and Webb: 1961). 

Training evaluators is crucial to the evaluation process. 

The following procedures have been used for improving the 
performance of those responsible for evaluation of teachers: 



Suggestions 
for Training 
Evaluators 
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• elective in-service courses or courses at universities 

• entire principals^ meeting devoted to evaluation 

• general explanation given at principalis meeting 

• workshop or clinic from one to three days; including 
assistance from an outside consultant, observation of 
films of live classrooms, co-rating for reliability, 
discussions 

• written manuals describing procedures, explaining forms 
and policy 

• central office personnel working individually with 
principal 

In systematically acquiring information to assess the 
effectiveness of the teacher evaluation system, an answer 
to these general questions must be sought; 

• Is improvement of instruction occurring? 

• Are teachers given information and resources to improve? 

• Are students learning? 



• Are teachers with problems showing growth cr being 
selected out? 

• Does the teaching staff understand how they are 
accountable? 

Because these questions are complex and difficult to answer, 
data about the teacher evaluation system must be as system- 
atically sought as the data collected on individual teachers. 
In addition, the means for gathering information about the 
evaluation system should be planned concurrently with the 
development of the teacher evaluation system . 



Analysis of 
Information 



If analysis of the data indicates a modif icatior: is needed, 
the revision process begins at the point where change should 
take place. For example, where data collection procedures 
are inadequate, then a review of purposes is not needed; but 
rather one should change the data collection procedures . 
Assessment may indicate that princ.ipals need more training 
in observation procedures or in methods of feedback to 
teachers , 
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AssessinenL of tiecicher evaluation is not a closed system. 

It occurs within the larger context of the total operation 
of schools and within the context of the local cominunity 
and society. 

CAUTION 



When school districts do not consider the 
multiple contexts within which the teacher 
evaluation system operates^ they increase the 
probability of ignoring important considera- 
tions in planning for problems tlvat may occur. 
These contexts include: 

0 human relations with teachers and 
cormunity 

0 development of school policy 
0 teacher training 

0 teacher organizations and negotiations 
0 evaluation of other personnel 
0 total school program evaluation. 



Human Relations with Teachers and Community 



Because evaluation of teachers is a human process as much 
as it is an organizational program, continuous efforts must 
be made to develop reliable measures of effective teacher 
behaviors which relate to pupil outcome in specific situa- 
tions. Pupils , parents , and other community members are 
at the very least indirectly involved in teacher evaluation 
and school districts must help them understand just how and 
to whom teachers and administrators are accountable. 



Development of School Policy 

School policy, developed by local schcol boards, is what 
gives direction to administrators. It is the public 
expression of a philosophy. Administrators must not only 
adhere to perconnel policies which reflect that philosophy, 
but professionals have an obligation to inform and interact 
with the public ard thus provide means to policy change. 
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Teacher Training 



Summary ; 
Assessment 
Evaluation 
Process 



Teacher evaluation seems to be increasingly more inter- 
related with teacher training. As new certification 
standards and intern-type training programs evolve , the 
new information about teacher effectiveness must be fed 
back into the personnel policy development effort of 
local school districts. 



Teach Organizations and Negotiations 

No longer do teacher organizations divorce themselves from 
issues involving evaluation and accountability. Many 
teacher organizations have already acquired negotiation 
agreements with local boards. State and national organi- 
zations ..ave obtained necessary legislation which gives 
teacher organizations bargaining powers. 

Regardless of one^s stand on the issue of teacher rights, 
and whether or not a labor-management relationship will 
emerge between teachers and administrators, negotiations 
will occur and they will include discussions about and 
planning for improved teacher evaluation processes . This 
fact should give even more impetus to lo^ial research 
efforts regarding effective teaching and systems for 
evaluating that teaching . 



Evaluation of 0 ther Personnel 



Systems for evaluating teacher effectiveness are, of course, 
related to the quality of those who administer that system 
and to the quality of those who are part of it (secretaries, 
cooks, etc.). No less rigorous evaluation procedures should 
occur, and consequently no less comprehensive personnel 
systems should be developed for school personnel other than 
teachers. Some belxeve that only when systematic evaluation 
of evaluators (e.g., principals, supervisors) occurs will 
teachers more readily accept accountability functions. 



Total Program Evaluation 

The evaluation of teachers is part of the larger efforts a 
school system makes in assessment of the total program. For 
example, changes in curriculum, groupings of pupils, school 
plant design, and instructional materials have an effect 
upon and are affected by teacher evaluat‘'*on. 

The evaluation of professionals provides information needed 
of to judge effectiveness of the individual teachers and allows 
better judgments to be made about individual modifications 
in training and placement. The assessment of the evaluation 
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process gives the information needed to make judgments about 
the effectiveness of that system, including how well the 
system : 



measured te.acher goodness 
planned the process 
® implemented the system 

o trained and supervised those who are evaluating 
teachers . 

Even if teacher effectiveness is measured carefully by 
process and product , and even if evaluators have been 
trained to observe teachers, only when the assessment of 
that process is precise and systematic will teacher evalu-" 
ation contribute fully to the total enterprise of education. 



Summary 

' The evaluation of teachers may serve many purposes: 

e to improve teach ing 

I 

& to reward superior performance 

© to supply information for modifying assignments 

« to protect either the individual or the organization 

' In legal matters 

« to validate the selection process 

® to provide the basis for planning for individual 
growth and development . 

; Improvement of instruction is considered by most educators 

^ to be the most important purpose for evaluation. However, 

the other purposes mentioned above must be considered in 
designing evaluation procedures. 

I The potential benefits of teacher evaluation are many; 

I however, there are possible adverse effects whicn the 

evaluator should consider when designing evaluation plans: 

f 

[ ® the conflict when both the function of accountability 

and assistance are expected in the same evaluation 

' © the tendency for the teacher to be shaped by the 

measurement lev ice used in the evaluation . 
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Once the purposes for the evaluation effort are established, 
the subsequent move is to develop precise statements which 
indicate what teacher performance* and learner outcomes are 
desired, i.e., what criteria will be used as standards to 
evaluate teachers . In determining what behaviors and out- 
comes are important in evaluating a teacher , the evaluator 
should recognize that multiple criteria must be established 
for each specific teaching position and these criteria must 
be reviewed frequently to assess their appropriateness. 

To answer the basic question of "What is important in 
teaching? these subsequent questions are in order : 

• Who will decide what is important? 

• How will the information be acquired for deciding what 
is important? 

• How will this information be gathered and analyzed? 



In answering the first question, it is suggested that in a 
school district, criteria decisions will be enhanced if 
judgments from a jury of authorities (teachers , administra- 
tors , parents , students , etc . ) are pooled . Such a coopera- 
tive effort gets away from one person or a biased group 
making the decision and often has a positive effect on 
morale. In terms of the second question, many procedures 
can be used for acquiring information needed for criteria 
selection. Probably the most effective means is to have 
individuals conduct a position analysis for each position 
so a detailed description of what is important can be 
obtained. Analysis of the information obtained will be 
improved if some empirical, rather than "arm-chair’^ method 
is employed . 



In planning for data acquisition and analysis, major consider- 
ation must be given to the measuring instruments to be employed 
since it is recognized that evaluation is highly dependent 
upon measurement as a basis for information gathering. 



Consequently, selecting an adequate and competent measure is 
crucial , and the evaluator should consider the following 
practical restraint s : 






o 



0 






the cost of acquiring the data 
the time it will take to gather the data 
the availability of adequate data sources 
the resistance of sources to evaluation. 
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In addition, the evaluator should consider the following 
questions : 



o Is the instrument relevant ? 

• Is it reliable ? 

• I s i t valid ? 

© Is it similar to the actual criterion performance , 
i.e., what is its fidelity ? 

• I s i t easy to administer ? 

The measuring instruments used for gathering data for teacher 
evaluation usually can be categorized by identifying whether 
the purpose of the evaluation is to assess the process of 
teaching (the teacher *s performance in and/or our of the 
classroom) or the outcome of teaching (the learner’s develop- 
ment following instructional treatment). In both categories, 
measuring techniques range from systematic procedures (e,g,, 
interaction analysis systems used by a trained observer) to 
non-systematic procedures (e,g,, the casual observation by 
a principal) . In determining which procedure is best , the 
evaluator should recognize that the procedure to be employed 
should be directly related to the kind of information desired. 

In bo th business and schools . some kind of meeting between 
supervisor and employee often follows observation of the 
subordinate’s work. This meeting is referred to as an 
evaluation conference by educators , and as a performance 
appraisal interview by business personnel. Essentially, 
both are used to fill the same purposes. 

In holding post-observat ion conferences , it is recognized 
that a teacher more readily accepts decisions where the major 
focus is on improving rather than fault-finding. In terms 
of the availability of the information gathered from the 
observations and conferences, open personnel files and reports 
to the Boards of Education tend to reduce anxieties and rumor 
among those being evaluated , 

Any system of evaluation of teachers should be periodically 
analyzed for problems. Some evaluation problems may not 
derive from the system itself, but may be due to problems 
of implementation . Too often, after a system has been 
developed for the evaluation of teachers , schools fail to 
develop a systematic approach to judge the effectiveness of 
that system. 
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In assessing the evaluation process, analysis of diff 
necessitates examination of; ^-^ysis ot diff 

• the realism of goals 

• the adequacy of their implementation 

• the effectiveness of the teaching procedure. 
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Chapter 5 



Purpose 



Major Topics 



Significance 

of 

Perspective 



TRENDS AND MAJOR IDEAS 



This chapter presents trends and major ideas found in 
practice and research on selection and evaluation of 
teachers. The purpose is to re-examine the ideas collec- 
ted and organize them in a way that helps to establish 
their significance. The previous chapters have presented 
a series of detailed ideas by relating them to sequences 
of selection and evaluation activities as they occur in 
practice. However, this chapter presents a different and 
more global perspective of the selection and evaluation 
process , examining themes which tend to unify the ideas 
and discussing trends which appear to be significantly 
affecting research and practice. 

Six major ideas seem to permeate both research and 
practice in the selection and evaluation of teachers. 

They are: 



• systems analysis of total personnel procedures ; 
a consideration of relationships of selection, 
evaluat ion, and supervision 

0 complexity of relationships among various aspects 
of selection and evaluation; a rejection of simple 
relationships as a means of explaining complex 
phenomena 

9 management by objectives 

9 emphasis on self evaluation 

9 the product-process relationship, i,e,, the 

relationship of outcomes and the procedures used 
to produce them 

9 management of information in a systematic fashion • 

\^^hy is it important to view ideas from a more global 
perspective? Basically , the advantage of such a 
perspective is to provide an alternative way of looking 
at ideas in order to help determine their significance. 
As one becomes enmeshed in the details of particular 
findings and specific procedures, he can become so 
engrossed in examining the details of research reports 
that he fails to raise the question of whether the 
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\>Jhy 

Examine 

Total 

System? 



research should have been conducted in the first place; 
so interested in understanding the way an administrator 
has solved one problem that he overlooks another which 
should have been solved but was not . 

The details must be examined, of course, and questions 
must be raised regarding what someone else has written 
or said: what are the bases of the statements made; are 

they knowledge or opinion; are the conclusions justified 
on the basis of the evidence; how generalizable are the 
results; how much redundancy is there between this state- 
ment and those made by others? Also, there is the prob- 
lem of determining what general conclusions may be drawn 
and how they may be communicated to others . 

But examination of such details leaves certain essential 
questions unanswered; for example: What themes unify the 

related ideas? I^at trends are significantly affecting 
research and practice? What major ideas represent the 
essence of multitudinous activities and their underlying 
motivations? 

As one searches for trends and major ideas, he is affected 
by his background and current surroundings; but he is 
influenced by new information also. Where the flow of 
ideas is exceptionally strong, he is forced to reconsider, 
to expand his views, and to synthesize the new with the 
old ; he is stimulated to adopt a broader perspective • 



In many scientific areas, the development of knowledge has 
proceeded by subdividing large areas of investigation, 
studying the smaller units intensively, and then synthe- 
sizing the findings from individual investigations. This 
approach has developed specialists who have vested inter- 
ests in particular small areas of investigation and has 
created a situation in which it may become difficult to 
determine the relationship of the special areas to the 
larger scientific area. 

In a similar fashion, administrative practices in large 
organizations tend to subdivide labor and create special- 
ists with narrow views of the total organization’s 
operation. As time ps.sses, subunits carry out rituals 
that may contribute very little to the other subunits or 
to the productivity of the organization. Systems analysis 



Personnel Systems: The Relationships of 

Selection, Evaluation, Supervision 
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procedures have helped to correct some of these problems 
by emphasizing the need to understand fully, through an 
examination of the total system, how the organizational 
subunits function. 

CONCLUSION 



The systems analysis approach has affected, 
the examination of school personnel systems 
and has precipitated discussions of the 
relationships among the functions of selec- 
tion^ evaluation^ and supervision and in- 
service training. (practice: education) 



The systems approach is particularly significant when one 
understands that the results of an excellent selection 
process can be nullified by poor supervisory treatment or 
ineffective inservice training programs. Also, if the 
evaluation process is inadequate, one might never know 
how effective the selection process was or what type of 
supervision or inservice training should be used. 

SUGGESTION 



School districts should perceive selection 
and evaluation procedures as part of a total 
personnel system which also includes super- 
vision. (viewpoint: research team) 



Relation- Figure 5.1 illustrates the relationships of these three 

ships components of the personnel system. It will be noted 

that the selection process is preceded by recruitment, 
or the development of an adequate applicant pool. Once 
an individual has been employed, he is given an assignment, 
with certain tasks to perform; the assignment also in- 
cludes supervision , inservice training and evaluation. 

In Figure 5.1, the arrow from "select in" to "supervision 
and inservice training" indicates that, at the time of 
selection, a recommendation might be made that the 
person receive a particular type of supervisory assistance 
or be enrolled in a special inservice course. 

Within the "assignment," information from "evaluation" 
flows in three directions: to "supervision," indicating 

that evaluative information can assist in changing the 
nature of a person’s assignment or his supervisory 
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Figure 5.1. Personnel Systems: Selection, 

Evaluation, Supervision. 
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treatment; to the ^’select in" box, indicating that 
evaluation provides a basis for assessing the effec- 
tiveness of the selection process; and to the "select 
out" box, indicating that some individuals will be 
released from their assignment on the basis of evalua- 
tion. When a person is released, an exit interview is - 
conducted to acquire information which will be used to 
improve the selection, evaluation and/or supervision 
functions . 

Figure 5.1, by illustrating the relationship among the 
three major functions of personnel, shows how the total 
personnel system can be analyzed for flow of information 
and for elements of the system which might cause errors. 
One may also examine the system from a decision and 
research viewpoint. 

Decision Based on decision theory, one might examine a situation 

Making from the viewpoint illustrated in Figure 5.2. A situa- 

tion exists and the decision maker asks a question 
regarding what is likely to happen in that situation. 
After action has been taken, the decision maker is then 
able to ask another question, viz., "what did happen?" 




Figure 5.2. A Simple Decision Situation. 



This simplified diagram illustrates the points at which 
the decision maker asked two questions, but what is the 
function of the questions? When the decision maker asks 
what is likely to happen (Step 2), he is really trying to 
make a prediction on the basis of the information avail- 
able. When he asks what did happen (Step 4) he is evalu- 
ating the consequences of his action. Figure 5.2 can be 
modified to indicate the nature of decisions at Steps 2 
and 4 by naming the types of question being asked. (See 
Figure 5.3.) 
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Predict 



Evaluate 



Figure 5,3, Two Types of Administrative Decisions: 
Prediction and Evaluation » 



The 

Research 

Function 



But predictions and evaluations can not be made without 
information, A person has two choices regarding how he 
will make decisions: (a) he can make decisions (either 

predictions or evaluations) solely on the basis of infor- 
mation available at the time the decision needs to be 
made, or (b) he can systematically collect further infor- 
mation to make these decisions. If he chooses the latter 
option, he performs a research function. Figure 5,3 can 
be modified once more to indicate at which points the 
research function is performed. 




Research to Predict Research to Evaluate 

& decide 

Figure 5,4, The Contribution of Research 
to Administrative Decisions • 

At Step 4, we have added * Mecide , " This shows that the 
decision maker evaluated what did happen and then 
decided what action to take. We can now draw Figure 5,5 
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which summarizes selection and evaluation procedures from 
a decision making viewpoint. 



1 




Figure 5.5. An Illustration of How Research Contributes 
to Making Selection and Evaluation Decisions. 



191 



204 



cj> SUGGESTIONS 



Simple 

Model 



The following suggestions^ are made on the basis 
of analysis of Figure 5.5. 

0 For a gersonnel system to fu720tion 
effectively , research should he an 
integral part of the system. 

0 Research should be conducted for two 
reasons at two different times: to pre- 

dict conseqi^ences for selectiori decisions, 
and to evaluate results of actions so that 
evaluation decisinns may he made. 

0 Research to evaluate actions should he 
co7^ducted concurrently with the actiori 
taken rather than on a post hoc basis. 

(viewpoint: research staff) 



Complexity of Relationships 



For some time, research on teacher effectiveness was 
conducted on the basis of a relatively simple model, 
consisting of a single measure of teacher effectiveness 
(a criterion) and its relation to a group of potential 
correlates (predictors). Figure 5.6 illustrates this 
model . 



Measures Prior to 
Teaching 




Measure of Teacher 
Effectiveness 




(Predictors ) 


Correlation 

Indicates 

Relationship 


(Criterion) 


e. g . , Grades in 
College ; Ratings in 
Student Teaching; 
Attitude toward 
Teaching . 


e . g. , Ratings by 
Principal ; Peer 
Ratings ; Student 
Achievement . 



Figure 5.6. A Simple Model for Conducting 
Research on Teacher Effectiveness. 




Many times the criterion consisted of a rating hy a 
principal or supervisor , although ratings by students 
or peers and measures of gain in subj ect mastery were 
also used. The object seemed to be to determine ’’Wliat 
makes a good teacher?^* The assumption was that some 
universally acceptable criterion of t eacher effective- 
ness was available and, therefore, could be predicted 
and controlled . 

This view also permeated practices in the profession, 
and many administrators were unwilling to consider how 
the unique behaviors of teachers related to specific 
teaching situations. Consequently, these administrators 
chose teachers on the basis of characteristics they 
considered important for all positions, resisting trans- 
fer of an unsuccessful teacher to a new situation on the 
grounds that a good teacher could do a good job regard- 
less of assignment . Some teachers can; however , not 
all do. 

CONCLUSION 



Teachers behave differently in some 
situations than in others and a given 
set of teacher behaviors will affect 
one group of students differently from 
another group; therefore^ a teacher may 
exhibit more effective behavior in one 
situati.on than in another. (research: 
Gage^ 1963; Flanders^ 1969) 



Recent Recent research and practice indicate that the 

Approaches significance of situations to teacher behavior and 

the effect of different teacher behavior on students 
has been recognized . Consequently, current models 
used to study and explain classroom learning are more 
complex (and more realistic) than prior models. The 
newer approaches take into consideration more aspects 
of the situation and of teacher behavior in attempting 
to explain why students learn as they do ; they have 
abandoned a search for teacher behavior which "works" 
in all situations for teacher behavior which appears 
to be productive in similar situations. 

Current emphases in research have precipitated study 
and discussion of the following: 

• use of several measures of teacher effectiveness 
rather than a single overall indicator 
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• relationships among these various measures of 
teacher effectiveness 

• improvements in the judgment process used in 
evaluation 

9 satisfaction of pupils, teachers, or adminis- 
trators as measures of teacher effectiveness 

• relative stability and reliability of data 
regarding behavior and characteristics of 
teachers. 

Within the framework of selection research, the new models 
have discouraged the attempt to answer scientifically the 
question of what is ultimately desirable in teacher 
behavior. Rather, the present models are based on the 
idea that it should be possible to study the relationship 
between particular teacher actions and the results of 
these actions when people in the organization have 
particular values . 

A formal examination of the decision process, as it 
relates to selection, forces attention toward the prob- 
lem of rigorously identifying what it is that determines 
people’s choices. Some selection personnel behave as if 
they have solved this problem by making their process 
somewhat less formal. A moment’s reflection will surely 
indicate that such is not the case — probably they have 
repressed the problem rather than solved it. Their 
ostrich-like behavior allows the question of ’’What will 
be our basis for choice?" to re-emerge every time new 
information is obtained or new applicants apply. Selec- 
tion personnel must have some basic means to match person 
and position. A lack of specific correlations between 
measures available prior to teaching and measures of 
teacher effectiveness (i.e., validity coefficients) for 
their specific situation should not cause personnel 
people to take evasive action, however. 
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Lcoat school systems should not ignove the 
problem of specifying bases for selection 
choices. Districts may begin by specifying 

0 necessary teacher behaviors 

0 desirable teacher behaviors 

outcomes expected to be precipitated 
by these behaviors 

The information specified can become the 
basis for later evaluation of the selection 
decision. (viewpoint: research staff) 



As necessary and desirable teacher behaviors are 
identified, care should be taken to insure that they 
represent the types of behavior that can be and have 
been observed in specific situations. They should 
not represent general after-the-fact impressions of 
an event, as such observations are notoriously 
untrustworthy and are likely to establish criteria 
that are biased in favor of factors such as general 
appearance, manner, and personal likeableness • ^ 



^Thorndike (1949) gives an example of a relationship 
between likeableness rating and overall rating of 
effectiveness in the U. S. Air Force. Although likeable- 
ness was explicitly rated low in importance, it corre- 
lated very highly with overall rating — indicating an 
implicit importance to the raters. This implicit 
importance probably resulted because no formal strategy 
required pooling of ratings in a specified manner that 
would have prevented the overall impression from unduly 
influencing the specific observations. 
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Management by Obj ect i ves 



Assumptions 



Steps 



The tremendous appeal to businessmen of the **managemc*nt 
by objectives'** concept has been transmitted to scliool 
leaders. The basis of the appeal is simple: the con- 

cept offers the possibility of greater accomplishment 
and higher employee morale . 

The concept consists of tv;o basic assumptions: 

• The clearer the idea one has of what he is 
trying to accomplish, the greater the chances 
of success. 

• Progress can be measured only in terms of 
one*s goals,** 

From these two ideas, a variety of practices has developed, 
most of them variations of the following steps: 

• setting objectives in terms of expected results 

• working toward these objectives 

• reviewing progress toward the objectives. 

The flow diagram in Figure 5.7 is one illustration of 
steps that might be followed in a "management by objec- 
tives" procedure . 



*The term was first used by Drucker in Practice of 
Management , 1954. See also Odiorne’s work (1965) 
and that of the National Industrial Conference Board 
(1968) . 

**Although two terms, goals and objectives, are used 
almost synonymously in this presentation, it is recog- 
nized that many writers use the term "objective" onlv 
when referring to behaviorally or operationally defined 
outcomes, reserving the term "goal" for more global 
outcomes. 
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Figure 5.7. Diagram of "Management by 
Objectives" Procedures. 
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Usually Steps 1 (examining the situation) and 
(goal setting) are performed at more than one 
level in an organization* For example, in a 
school district, the 



and the 



transmit information 
regarding policy, a 
general view of the 
future, and the goals 
of the total school 
district to 



so they have a meaning- 
ful context within which 
to formulate appropriate 
school building goals 
with a 



The school building 
goals then become a 
basis for 



to set specific obj ec- 
t ives for teaching in 
cooperation with 
principals. 




2 



Description Generally, the system of management by objectives 
can be described as a 

process whereby the superior and subordi- 
nate managers of an organization jointly 
identify its common goals, define each 
individual’s major areas of responsibil- 
ity in terms of the results expected of 
him, and use these measures as guides 
for operating the unit and assessing the 
contributions of each of its members . 
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Suggestion 



(Odiorne, 1965:56). Because of the nature of manage- 
ment by objectives, the concept is often linked with 
the term ’’accountability/* * Individuals are "accountable” 
for their contribution to the accomplishment of certain 
outcomes . The responsibility of individuals in the 
organization is not only to superiors, howevers. 

SUGGESTION 



It is the subordinate's responsibility to 
determine what resources will be needed 
to meet the agreed-upon goals and to inform 
his supervisor what they are; it is the 
supervisor ^s responsibility to see that 
these needed resources are made available. 
Each has a role to play in seeing that the 
subordinate's goals are achieved. (practice: 
indus try) 



In a school district , the individual teacher might 
draft his own proposed objectives, but within a 
context tending to keep his proposals consistent 
with the needs of the total organization. Approval 
of individual proposals by the principal keeps the 
objectives of the teachers tied into the mainstream 
of the system. 

Each teacher would then be evaluated in terms of 
whether he meets the objectives he has set. Although 
the individualized aspect of the evaluation would 
require less structure than systematic analysis of 
classroom procedures , establishment of checkpoints 
where results would be measured and discussed would 
tend to facilitate a more continuous evaluation 
process . 

If it is desired to use management by objectives 
procedures, the following points should be con- 
sidered : 

• Agreement on goals should be reached at 
linkage points in the organization (e.g. , 
between the superintendent and principals , 
or principals and teachers), since commit- 
ment to goals is essential to make the 
concept work. 

• Each person should understand the plan, 
know for which part he is responsible. 
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what results are expected, and how and 
when these results will be measured. 

• The manner in which management by objec- 
tives procedures are initiated and used 
(e.g., in autocratic fashion or not) 
varies with organizations and affects 
morale and acceptance of the plan by 
employees. 



Wliereas the management by objectives system has been 
developed because of a concern for managerial con- 
trol mechanisms based on agreed-upon goals, the 
emphasis on self evaluation has a different motiva- 
tional base. First , there is a reasonable 
assumption that teacher behavior changes most 
readily when a teacher is provided objective data 
regarding his own teaching. Since raw data are 
perceived by the teacher as **more objective" than 
data which have been analyzed and/or interpreted 
by an outsider, self evaluation begins with an 
audio or video tape recording of teaching behavior 
to be analyzed by the teacher himself . The teacher 
is then helped to develop ways to code and analyze 
the recorded behavior for the purpose of making 
judgments regarding its utility. 

Second , the teacher ’ s analysis helps to reduce the 
natural conflict that is often encountered when an 
outsider makes judgments about teacher behavior. 
Since the supervisor is placed in the role of a 
resource person to provide assistance in developing 
coding and analysis skills, he is no longer per- 
ceived as a threat to the teacher . 

CONCLUSION 



The common goal of the supervisor /principal 
and teacher in self evaluation is to provide 
a teacher the opportunity to improve his 
teaching skills by observi:'ig his own behavior 
in a threat-free atmosphere. (position) 



The third base for self evaluation concerns the role 
of the teacher as a professional and whether he should 
be autonomous with regard to his own improvement. In 
effect, there is an expressed desire on the part of 



Emphasis on Self Evaluation 




the teacher to ]^e the determiner of whether process 
goals were met ind to decide on appropriate action 
to be taken. Examined in the light of Figure 5,7 
(repeated from the preceding section) , there is a 
desire for the teacher to be independent of outside 
supervision during steps 4, 5, and 6, Self evaluation 
tends of emphasize the analysis of procedures at Step 4 
rather than the outcomes of the procedures. 
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Figure 5,7, Diagram of "Management by 
Obj ectives" Procedures , 



Other 

Procedures 



Student 

Feedback 



Self evaluation can deviate from the description given 
thus far, however. Films or audio recordings can be used 
cooperatively with principals or with other teachers for 
the purpose of gaining insights into the meaning of 
teacher behavior and student responses in classrooms , 

In fact , self evaluation can evolve from regular external 
evaluations, where the external evaluator is adept enough 
to encourage and stimulate continuous self examination by 
his day-to-day contacts as well as his formal evaluations. 



SUGGESTION 



External evaluation of teaohers should 
enaourage self evaluation procedures and 
the continual diagnosis of individual 
teacher behavior. (practice: schools) 



Similarly, structured or informal feedback from students 
can provide information that helps teachers to see how 
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lu-l.^tv’iur be clianged (Gage, 1963; Mcrllno, 

19()3). in t , feedback from students is one source 
of i ni t»i inaL ,i u'u i bat offois great potential for teacdiers 
i n su If evul uat ion process , 

fi.L'dback from st. udunts is: 

y* rcl.alivuly easy to acquire 

gene r .■ 1 1 1 y i I i a b I e 

lias f ace v^a.l i d i t y 

AJ.L hoi.igli .->ome r^'seart.h iiaij [jeen done in this area (and 
some teacliers use siudeiit feedback frequently) very 
little luis l>et:n dc.nie to train students to be more sojdii..-- 
ti Gated observers in order to provide more complex data 
for teachers. 

ShGCdGSTTON 



2\:u<:hcvs rJiould he e'ncouraged to acquire 
feodhaek from suidents as a regular part 
of self cvaliiutiori procedures. (practice: 
schools) 



Product -Process Relationship 



iHsUrry Tlie recent emphasis on accountability has tended to focus 

attention on tiie ’’product,” i,e,, evidences of student 
learning. However, the emphasis on accountability was 
preceded by a considerable period when most research 
stressed investigation of procedures used in the class- 
room, and more recently, self evaluation of classroom 
interaction. Therefore, it is natural to consider evalua- 
tion of both the products of teaching and the procedures 
used to effect these outcomes in specific situations . 

Some districts are beginning to design systems to study 
both procedures and outcomes, believing that measurements 
of both are necessary to modify classroom instruction in 
any sysi. ema tic and meaningful way , 
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An Analogy The system for establishing a product-process relation- 
ship in a specific situation can be easily understood by 
means of an analogy. Suppose a person decided that he 
is 15 pounds overweight and would like to remedy the 
problem. He decides that the cause of his problem is 
poor eating habits. He sets a goal of losing 15 pounds 
in 15 weeks and a short-range goal of f ive pounds in 
f ive weeks , His method for lo sing weight will be to 
reduce his morning and evening meals and to eliminate 
snacks which have more than a small number of calories. 
An analysis of the problem might be as follows: 



Problem: 

Short-range goal: 
Long-range goal: 
Subsidiary goal : 



overweight by 15 pounds 
lose 5 pounds in 5 weeks 
lose 15 pounds in 15 weeks 
modify eating habits 



Method : 


breakfast : 


reduce by 1/3 to 1/2 




lunch: 


no change 




dinner : 


reduce by 1/3 to 1/2 




snacks : 


raw vegetables or 






fruit, diet drinks 


Control : 


Weigh weekly and record on 



chart , If behind schedule or 
if no loss in three consecutive 
weeks, modify method by either 
reducing intake or increasing 
exercise. 



This person now has a way to determine whether a particu- 
lar method will produce certain results — for him , in his 
situation . 

If results are not as planned, he must raise certain 
questions: (a) were the goals reasonable or did he try 

to lose too much too fast? (b) were the procedures 
implemented as planned? (Or did he sneak some extra 
snacks and fail to cut down at his evening meals?) 

If the goals still seem reasonable and if the procedures 
were properly implemented, then on the basis of the 
relationship between the process and the outcomes he is 
forced to conclude that the procedure did not work — for 
him ^ in his situation # Note that he does not generalize 
the value of his method for other people; his conclusion, 
based on the data collected regarding a product-process 
relationship, applies only to his own case. He can now 
take appropriate action based on a predetermined strategy 
for his particular situation. 
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CONCLUSION 



A}ialuGis of product-proaesG vclatioriGhip 
allows conolusions to be reached regarding 
the effectiveness of procedures in specific 
si tuatiois, ( viewpoint : research staff) 



Flow of Figure 5.8 shows a generalized network for personnel 

Activities evaluation of the product-process relationship in a 

specific school situation . Steps 4 - 7 (results) are 
probably the only steps which need any elaboration . 

Step 6 indicates that the results of the action taken 
are examined in terms of both product and process. 

This is important since often one of the results is 
ignored. The product is measured following the action 
taken (Step 4) but Step 5 indicates that the process 
is observed concurrently with the action taken. 



CONCLUSION 



Output and pvoaeduval goals are move likely 
to be understood and aaaomplished when they 
are developed cooperatively by the teaaher 
and principal (or supervisor) and are written 
in specific terms. (viewpoint: research 

staff) 



It hardly seems necessary to emphasize that.^ in the field 
of ed^’cat ion , measurement of both product and process 
requires considercible development of measurement pro- 
cedures and record keeping systems , 
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Figure 5.8. Personnel Evaluation: Product- 

Process Network. 
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At Step 7, the question of whether the results are 
satisfactory is asked; e.g., in the case of our over- 
weight dieter, this is analogous to asking whether the 
desired amount of weight has been los t . Note that throe 
possible sources of difficulty can be identified if the 
goals have not been met: 



o the goals themselves 

• the procedures , and/or 

• their implementation . 



If a combination of these items is causing the proolem, 
a solution will be difficult unless possible causes are 
eliminated by systematically changing one of the items 
and recycling through the process . 

Two additional features of the network are important. 
First , the conclusions are based on data collected in 
a specific situation ; there is no attempt or need to 
generalize to another situation . Second , the duration 
of the cycle can be short, and a short cycle helps to 
correct problems quickly'. 

CONCLUSION 



A personnel evaluation system for analyzing 
the produot-prooess relationship allows one 
to : 

0 determh^e whether a particular teaching 
metliod produces results in a specific 
situation 

0 determine whether poor results are due 
to inappropriate goals, poor implemen- 
tation, and/or an ineffective procedure 

0 recycle events quickly to remedy problems, 
(viewpoint: research staff) 
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Management of Information 



Significance An analysis of selection and evaluation of teachers 

indicates that both require the making of many decisions: 
selection requires cho ices among applicants and a 
det ermination of when to make offers ; evaluation 
requires that actions be taken to improve supervision 
and in-service training , to modify assignments , and to 
change selection procedures . In order to make these 
important decisions, it is necessary to manage infor- 
mation efficiently. The management of information 
includes : 

• acquisition 

• processing (i.e. , storage, consolidation, 
retrieval , display) 

• analysis and interpretation. 

CONCLUSION 



quality of teacher selection and evaluation 
decisior.s depends on the way in which infor- 
mation .'sed in the decision process is managed, 
(viewpoint: research staff) 



Purpose 



\ 

I 

L 



The purpose of examining the management of information, 
then , is to determine whether established procedures 
facilitate the making of decisions in an efficient 
manner. Three outcomes are desirable: 

f good decisions 

• quick decisions 

• economy. 



because of differences in size of operation 
bility of funds and personnel, various ways 



and availa- 
of managing 



i 
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information have been developed and found acceptable for 
particular situations . 



Problem 
Areas : 

Se lection 



} 

I 

Si 
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However, certain problems are inherent in managing 
information in the selection and evaluation of teachers, 
and should be considered before modifying procedures. 

For example , in examining the selection process, the 
following are often major problem areas: 

Acquisition of information . At least two trouble spots 
may occur in the acquisition of information. 

First , there is the problem of reliability, and this 
pertains to most of the information acquired. Since 
a considerable portion of the information is acquired 
through self-report devices (such as application forms 
and placement office forms) , recommendations , and 
interviews, the concern for reliability is well founded. 
The most common procedures used to increase reliability 
of information are: 

• cross-checking information from ivritten documents 
for discrepancies 

• asking references to respond to specific questions 
and comparing information from several references 

• i.iaking telephone calls to references to probe and 
to verify information 

• training interviewer s to acquire specific 
information. 



Second , there is the problem of obtain 
mation. This may seem innocent enough 
combined with other problem areas, e.g 
information, it can function as an inf 
and affect the effectiveness of decisi 



ing too much infor 
However , when 
. , display of ‘'.he 
orma t ion overload 
ons . (Bolton , 



.-96', p. 340) 




SUGGESTION 



Only information which will be used in the 
teacher selection decision should he included 
on application forms or placement office 
credentials. Any information^ which may he 
useful once a person is employed^ should he 
acquired after employment, (viewpoint: 
research staff) 



Consolidation of information , Wlien information is 
acquired from a variety of sources, e,g., letters of 
recommendation, the interview, evidence of extraclass 
activities, it sometimes is difficult to manage all of 
the bits of information. Consequently, it is beneficial 
to consolidate some of the information in order to deal 
with it effectively. 

For example , perhaps an applicant has had multiple 
experiences in working with small children (information 
obtained from application form); has a recommendation 
from a student teaching supervisor which indicates that 
children seek him out when they have difficulties and 
that he shows consideration for their problems; and 
indicated in an interview that his primary motivation for 
teaching is to help all children to develop and solve 
problems which they face— whether the problems be personal 
or academic. These three bits of information may be 
related and best handled by combining them into a measure 
of the person ’ s consideration of children, or an empathy 
factor. Inhere possible, one should look for ways to con- 
solidate information in such a manner that it can be 
related to the selection decision. 

Display of information . When information to be retyped 
by a person making a selection decision is displayed in a 
disorganized fashion on multiple written documents, its 
significance may be difficult to perceive. Therefore, it 
is recommended that local districts develop single 
summary documents which display information considered 
important to the selection decision. Since the importance 
of information acquired from various sources will differ 
from district to district, it will be necessary for each 
district to develop its own strategy for transfer of 
information to a summary document; however, when the 
summary document has been devised, more discriminating 
and reliable decisions are likely to result. 
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Retrieval of informat iori t Retrieval of information 
acquired regarding applicants is most important when 
specialized skills are being sought. For example, 
suppose that there are three vacancies in a large higli 
school English department. Suppose further that it would 
be beneficial if at least one of the three new teachers 
lias some background in speech and the dramatic arts. If 
there are a large number of applicants, the retrieval 
of information regarding course work and extraclass 
activities pertinent to speech and drama may become 
rather complex. Unless such problems are planned for , 
the system which is designed will not reduce the time 
lag or cost in retrieving information. 

In managing information regarding the evaluation of 
teachers, the following are major problem spots which 
should be examined . 

Acquisition of information . The problem here is similar 
to that faced in selecting t eachers , in that one is con- 
cerned with reliability of the information. Whether 
collecting information regarding an outcome such as the 
learning of pupils (a product) or the behavior of a 
teacher (a process), reliability is a formidable problem 
which is recognized by both practitioner and research 
personnel. Reliability can be increased by: 

• specifying precisely the information that is 
needed 

• using professional assistance in designing 
ways to acquire the information (or to design 
measuring devices) 

• training personnel to use well-developed 
observat ion procedures . 

Interpretation of information .. Regardless of how reliably 
information is acquired , if it is misinterpreted it has 
been mismanaged. Interpretation has to do with the mean- 
ing attached to information , and meaning of information 
regarding the evaluation of teachers is generally deter- 
mined by comparing relative amount s of behaviors , 
products , or activities . 

For example, if Flanders ’ Interaction Analysis technique 
is used to measure or describe* verbal behavior in the 
classroom, the first action of the observer is to record 
the verbal behaviors in terms of categories. The sequence 
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of behaviors* is then plotted on a 10x10 grid which helps 
to cluster those behaviors which were the same; this is 
an analysis phase. 

Tflien one tallies the frequencies of behaviors and makes 
comparisons among categories, or compares the percentage 
of the time spent in teacher talk with the percentage of 
time spent in student talk, he is then interpreting the 
information. The point to be made here is that inter- 
pretation is the attachment of meaning to information 
following reliable acquisition (or measurement) and 
analysis of the information. 

CONCLUSION 



Where the information to he used for evaluation 
of teachers has been unreliably acquired or 
inadequately analyzed^ it is to be expected 
that conclusions reached in the interpretation 
will be faulty. (viewpoint: research staff) 



It seems relatively clear that proper interpretation of 
information will be facilitated by: 

• reliable acquisition or measurement of events, 
products, and behavior 

9 analysis which clusters information into cate- 
gories which are considered to be important 

• a display of data which allows comparisons to 
be made easily 

• comparison of views by the teacher and evaluator 
to determine how the context af fects the meaning 
of the information. 

Transmission of information . The manner in which 
information on evaluation is transmitted to teachers, 
to central office administration, and to boards of edu- 
cation can have a tremendous impact on how teachers 
react to the total evaluation process. 
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CONCLUSK^N 




Rose) 



Where real ocnsideration is shown for teachers 
and where they are presented information which 
is helpful to them^ there is evidence that they 
welcome the evaluation process. (research: 



However, teachers tend to respond negatively to 

• fault finding without suggestions for remedy 

• categorizations (good, average, , , , ) which provide 
little diagnostic assistance 

0 generalities which appear to have little factual 
basis 

0 reports made in secrecy 

0 reports made without clear contributions to 
organizational goals . 

Retrieval of information . The problems involved in the 
retrieval of information acquired in the evaluation of 
teachers are primarily related to: 

0 long-range planning and determination of teacher 
needs 

0 special assignments to teachers 
0 promotions 

0 design of inservice training programs. 

Where information regarding teachers is not stored in 
such a way that it can be readily retrieved, any tasks 
involving these functions may encounter delays, costs 
for re-acquisition, and loxvered morale on the part of 
administration of teachers. 
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CONCLUSION 



Recommenda- 

tions 



l^fheve possible^ the design of systems for 
managing information acquired in the selection 
of teachers should anticipate the many needs 
for such information and design an adequate 
way to store and retrieve the information to 
serve those needs. {viewpoint: research 

staff) 



As officials in sch ol systems become involved in 
designing ways to manage information for the selection 
and evaluation of teachers, the following recommendations 
should be considered: 

1. Design the system for managing information in terms 
of : 

• purposes to be served 

e availability of funds, equipment, personnel 

• magnitude of information to be handled. 

It seems rather obvious that a small school district 
would have different funds, purposes, and amount of 
information to be processed than would a large 
district, and such factors will partially determine 
the type of system designed. 

2. Plan a comprehensive system for managing information, 
and install it in phases if necessary. For example, 
it may be that a district has initial needs for bio- 
graphical information on the teachers in the system, 
anticipates a need for evaluation data at a later 
point for long-range planning, and eventually will 
desire rather comprehensive information on appli- 
cants for new positions. If the system is designed 
with the total needs in mind, the total implementa- 
tion may be shorter than if the system is periodically 
disrupted because of inadequate initial planning . 

3. Consider incorporating the following in a system: 

• forms and written records 

• visible card systems 
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typos of folders to use 



• manual card sort equipment (which uses 
holes around the outside of cards and 
metal rods, or a light source which 
performs the same function as the rods) 

• data-pro cessing cards , tapes , computer 

• technical help 

• clerical help 



Summary 



Summarized in this chapter are six major trends or ideas 

1 . Examination, by a systems analysis approach , of 
the functional relationships among selection, 
evaluation, and supervision of teachers prevents 
isolation of any one function, 

2. The current complex models of selection and 
evaluation of t eachers , incorporating criteria 
that are multiple, dynamic, and specific to the 
situation are more realistic and useful than 
prior models, 

3 . "Management by obj ectives" can clarify organi- 
zational goals and show how these goals can be 
subdivided for action by work units and indivi- 
duals, Accountability is facilitated by these 
procedures , 

4. External evaluation can stimulate self evaluation, 
which leads to more continual diagnosis of 
individual behavior, 

5. Concurrent analysis of the results of product and 
process in a specific situation can be used to 
obtain feedback for correction of individual 
problems , 

6. The manner in which information is managed, i,e,, 
the information acquisition, data processing, and 
analysis and interpretation will affect the quality 
of decisions made in the selection and evaluation 
of teachers. 
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Obviously, these general trends or ideas are inter- 
related; incorporation of all six in a personnel 
system is not only possible but desirable. 
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Appendix 1.1 



Effect of ’’Halo Effect*’ on Correlations 
with Predictor Variables. 

Suppose one had a group of 12 teachers whose I.Q.’s ranged from 100 
to 140. Suppose further that he wanted to know whether this variable 
(intelligence as measured by the test which provided the I.Q. scores) 
predicted teaching success. If a measure of teaching success were 
available , it would be relatively simple to compute a correlation 
coefficient. Suppose it were decided to use the principal’s judgment 
of teacher effectiveness, as measured on a five point scale ranging 
from poor (1) to excellent (5). Let us further suppose that this 
principal allowed a general "halo effect" to influence his judgment 
and cause him to rate all of the teachers "5". The data for this 
situation could be arranged in tabular form as shown below for use 
in the correlation formula provided below the table . 



Table Al.l 



Fictitious Data for Computing Correlations 
between I.Q. and Teaching Success. 



:acher 


I.Q. (X) 


Success 


(Y) 


XY 






1 


100 


5 




500 


or 


5(100) 


2 


105 


5 




525 


or 


5(105) 


3 


110 


5 




550 


or 


5(110) 


4 


115 


5 




575 


or 


5(115) 


5 


120 


5 




600 


or 


5(120) 


6 


120 


5 




600 


or 


5(120) 


7 


125 


5 




625 


or 


5a25) 


8 


125 


5 




625 


or 


5(125) 


9 


130 


5 




650 


or 


5(130) 


10 


130 


5 




650 


or 


5(130) 


11 


135 


5 




675 


or 


5(135) 


12 


140 


5 




700 


or 


5(140) 


12 


EX = 1455 


EY = 60 


EXY 


= 7275 




5(1455) 














5X 


■elation 


coefficient , 


■r = — 




nEXY - 


EXEY 




xy / 


[nEX2 - 


1 1 

CM 


[nEY^ - 


(^y2] 



Examine the numerator for the formula. Since nZXY reduces to 5nEX, 
and since SXSY reduces to 5nZX (since EY reduces to 5n) , the value 
of the numerator is 5nEX - 5nEX = 0. Since the numerator of the 
formula is zero, the correlation coefficient is zero. 
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IL will 1 h> noted tlint this occured when tlie one measure was a const.int 
of 5. Regardless of what that constant (k) was, the correlation would 
have been zero (since the numerator would then have become kn/iX minus 
knl.X ) . This means that any time either of the variables in a correla- 
tion coefficient approaches a constant, the correlation itself v;il] 
approach zero. This explains the difficulty of obtaining significant 
correlations where errors of either halo effects or errors of central 
tendency occur. 
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Appendix 1,2 



An Analysis of Authentic and Phony Communication 
as it May Relate to Open Teacher Personnel Files 

What are the issues regarding open files and reports regarding teachers? 
Mostly they concern the authenticity of communications to teachers and 
about teachers. Suppose we consider two forms of communication, writ- 
ten and oral, and two degrees of authenticity, phony and authentic, 

(It is agreed that the dichotomy of the authenticity is forced and 
that authenticity is continuous, but the division into two parts 
serves the purpose of analysis and the conclusions are not distorted 
because of it,) Consider further that information could either be 
negative or positive (i,e,, in an evaluative sense, it could either 
be bad or good) , 

The different categories of information to be communicated could be 
classified as in Table 

Before examining the cells of Table Al, 2, let us assume that the oral 
communication regarding the teacher’s behavior is in direct face-to- 
face conversation between the teacher and the evaluator. Further, let 
us assume that the writter* communication is without the knowledge of 
the teacher, i.e,, written reports are not open to the inspection of 
the teacher. All communication is from the evaluator. 

Table Al. 2 

A Classification of Different Types of Information 
to be Communicated to Teachers. 

Written Communication 



Oral 

Communication 



Phony 



Phony Authentic 

neg, pcs, neg. pos . 
negative 1256 

positive 3478 



negative 9 10 13 14 

Authentic 

positive 11 12 15 16 



Let us now examine the cells of Table Al. 2. 
indicates that the oral communication to the 
negative, while the written communication to 



For example. Cell 1 
teacher is phony and 
the central office is 
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.ilso ]) 1 UM 1 V aihl nujviLivo. WIiaL would the results of sucli coinmun i r.i { h)ii 
likiMV \)c! Tho plumy and negative oral commun ica t itm would most 
1 i in.* 1 y j)i*eu i |M ta t‘e undesirable reactions on tlie part of tlie Learlu-r, 
(•’inre t lu- { n I (‘aaua t i on is pliony , wo could presume tliat tlic iKdiaviur 
is acLually desiral^le) wliilo tlie pliony written communication would \)k.‘ 
uiU‘ 1 ii i ci\ I , 

i‘‘h)le A1 . d presents an anaJysis of the likely results of various 
typrss ol, t:omiiuin Lent ion to the teacher and the central office. 



Table Al. 3 

All Analysis of the Likely Results of Various Types 





of Communication to 


Teachers 


Lc\l 


Likely Results 


Cell 


Likely Results 


] 


undesirable, unethical 


9 


non-existent 


2 


non -ex i s ten t 


10 


illogical , self-destrucL Ive 


] 


non-existent 


11 


illogical, destroys teacher 




evades issues 


12 


non-existent 


3 


nun-e'xi. s ten t 


13 


realistic 


6 


i 1 Log i ca 1 , destroys teacher 


14 


non-existent 


7 


evades Issues, dishonesc 


15 


non-existent 


8 


non -ex is ten t 


16 


realistic , motivating 


V rum 


the anaJysis in Table Al . 3, 


it appears that the following 



conclusions can be made regarding evaluation information, open oral 
comiminicat ion to the teacher, and closed written communication to the 
central ofiice. 

a. When both the oral and written communication are posit ive , 
then the results are likely either: 

• to be realistic and motivating, or 
o to evade issues. 

h. When b oth communications are negative , then results are 
e i thcr : 

• realistic, or 
o unethical . 

When one communication is positive and anoth er is negative, 
then results either: 

• are illogical , 

• are destructive to teachers , 
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• are self-destructive (to the evalua'or), 

« are dishonest, and/or 

• evade issues. 

The only two cells which offer promise for precipitating beneficial 
results are Cells 13 and 16, where both types of communication are 
authentic. Since the only types of communication with teachers or 
about teachers which are likely to precipitate beneficial results 
are those which are authentic in both written and oral forms, it seems 
reasonable to conclude that open personnel files (containing authentic 
information) would not be harmful to anyone concerned. Since open 
files do tend to reduce rumor and anxiety among teachers, it should 
be recommended that all written formal evaluations of teachers be open 
to examination by teachers. 



APPKNDIX 2.1 



Te.-iohor Se 1 ci: L i on and Iwa 1 iiaL i on 
(Jues t: i onnai rc 



De/i r Kespundon t : 

Tliis (jiiesL i onnai re has been designed so Chat you might respcnul it 
quick.iy and so that we might analyze the results and return tliem L.« 
Vi)u. VJe are particularly interested in recent practices or pnu'eclun' 
whicli you have found useful. Also, we are interested in any rosearrh 
that you iiave conducted on your selection and evaluation procorlures. 

Since our sample is relatively small and our project is on a tight 
scliedu.le, it is important to us that the questions be completed 
aunjrate ly and promptly. \\^e v^7ill appreciate it if you return this 
Fe bruary 5 in the enclosed self-addressed envelope. 

i’hank you for vour consideration and time in completing this 
(|uc*s t i onnai re . 



Sincerely , 



Dale L . Ho 1 ton 
Principal Investigator 



and Title of Respondent 



School Systenj 



Name of Super- 
intendent 



C i ty 



State 



Zip (]ode 



Check the enrollment group category of the scliool system: 



100.000 or more 

50.000 to 99,999 
25, QUO to 49,000 



1 16,000 to 24,999 4 

2 10,000 to 15,999 5 

3 less than 10,000 6 



Please return this questionnaire to; 

Select ioi'i and Evaluation of Teachers 11-4500 
309 MLlier Hall, College of Education 
University of Washington 
Seattle, Washington 98105 
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SECTION I: SELECTION OF TEACHERS 



r 



INSTRUCTIONS FOR SECTION I: 

Please indicate all individuals who are 
normally involved with a particular 
activity by marking one or more of the 
circles per line. 

Please relate the positions and titles on 
the right to equivalent ones in your 
school system. For example, what we have 
labeled '’supervisor^’ may have such titles 
as director, helping ":eacher, coordinator, 
etc. in your district. 



lA. DEVELOPMENT OF CRITERIA FOR 
SELECTION OF TEACHERS 

1. develops complete job description(s) 

2. identifies teacher behaviors to be sought . . 

3. determines information used to predict 

future teacher behaviors 

4. other , . 



/ 




. ^ 
o , 









.C ^ ^ ^ ^ # 

^ • . A o . . S' c/ ^ ^ Other 

iiiliio 0 oiliiHio 0 0 



er 



SiiitSiO 0 0 iiOiiSii 0 0 0 



iSiiSiiSiiO 0 0 ; 0 !iipi!ipii 0 0 0 
isiisiisro 0 0 idiildiildil 0 o o 



IB. RECRUITING 

5. determines employment needs 

6. describes the vacancies 

7. sends notification of vacancies 

8. develops recruitment brochure or other 

materials 

9. plans recruitment program 

10. conducts recruitment program 

11. makes contact with teacher placement offices 

and other employment agencies 

12. makes informal personal contact with potential 

candidates 

13. other . . 



iSiiSiijSilo 0 0 iGiiioijiPii 0 0 o 

ilpiipo 0 0 ipiiijiiii!! 0 o o 

Miiiio 0 0 iOiiiiiiiiiji! 0 o o 



iioiiiaiiiqii 0 o o ipiiipijipii o o o 

liliiliiipii 0 0 0 ipiiipiijiii; 0 o o 

iffiildilidH 0 0 0 idilidiiidiio o o 



!!P;!! 0 :iiOi: 0 0 0 iQii!Qii!Q!i 0 0 0 



iiPiiiPiOQii ° ° ° o o 

iioliiiioilo o o ioiiidiiioiio o o 



o 

ERIC 



IC. INFORMATION REGARDING APPLICANTS 

14. acquires information from application, tests, 

credentials, other documents . . . 

15. conducts interview(s) 

16. processes and analyzes information 

17. other . . 



iipiijPiijft 0 0 0 Sjijfiiiieiio 0 0 

WMM 0 0 0 iii|siis|io 0 0 

lliiiiiii 0 0 0 siiiiiiiio o o 

iiiiiillii 0 0 0 liilliiQlio 0 0 
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Section I: Selection of Teachers (continued) 



S 

. c- ^ 









10. STRATEGTKS AND DECISIONS 

18. determines relative importance of each 

item of information collected 

19. decides to collect more information .... 

20. assigns applicants overall rating .... 

21. determines rank ordering of applicant pool 

(or a portion of it) 

22. decides to reject applicant (s) 

23. decides to delay decision re employment . . 

2 ^. decides to offer contract 

25. officially decides to employ 

26. sends contract 

27. sends notification of vacancies filled to all 

applicants 

28. sends notification of vacancies filled to 

employment agencies 



C 0/ 

• O' 

C* 'C Qj f ^ ^ - , — 

( 2 / Q -0 O ^ *0 . . 

•>^y yA Qj <J ^ c* 

Qj <J ^ /O <j <J O Qj ‘IXz ^^LiU 
O Cn /Q Co Q 07 

^ ^ (Speci, 

iiiOiiiOiiit)! 0 0 0 ijaiiiOijgliio 0 0 



0 0 ollolilSilSilo 0 0 



0 0 0;;k;|D|:d;:o 0 0 

iijliliQliiQl 0 0 OiiSj;|iig| 0 0 0 

iiilljiOiie; 0 0 0 0 0 0 

liiliisliiidi 0 0 oiioii;®i|Ki 0 0 0 

iHffliffl 0 0 oliiiiillio 0 0 

0 0 0 ildilloiiioii 0 0 0 



0 0 OHSiiidiiioi: 0 0 0 



29. other 



iiiS;|i3;;S 0 0 0:iigiig:j|Qi:o 0 0 
iilspils 0 0 oiiigiiigjjaliO o o 



i:e. controls 

30. checks reliability and validity of 
information collected 



iiiiPiiiPii;® 0 0 oiiiSliidiiiSliO 0 0 



3l. makes analysis of sources of potential errors ii::::;::::!.;::.;: 

in the selection process iilldiildilldl 0 0 OilSilSlISi^O 0 0 



32. conducts formal training sessions for people 

involved in selection of teachers ;llldll:d;:id: 0 0 0 0 0 0 



33. other 



ipiijOiiiO; 0 0 Oljijiiioiiigi 0 0 0 



IF. OTHER ACTIVITY (specify) 
34. 



o 0 0 jidiiidllil 0 0 0 
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Section I: 



Selection of Teachers (continued) 



OTHER ACTIVITY, con't. 



35. 



With relation to the selection of teachers, please describe any new procedures 
that you have adopted during the last five years. (Please enclose any materials 
that are used with these procedures.) 



36. 



Please describ'^ any revisions you anticipate in your procedures for selection 
of teachers. (Enclose any pertinent materials that assist in your description.) 



37. With relation to the selection of teachers, please describe the practices you 
use that are most important, effective, or useful. (Enclose any pertinent 
materials that assist in your description.) 




(Please attach additional comments if this space is not sufficient) 

235 






248 



SK(,TION II: 



KVAI.UATION OI' ri’AUIKKS 



I IiASTki, i:i Min;; i-ou SKC'l'lON II : 

i 

\ {iidioaU/ :iJ.l individuals who a; e 

^ with a particular 
«irt ivily hv mai'kiiiK one or more of the 
oil 0.1 es pur line. 

u. rloa.se relate the positions and titles on 
the ri^'ht ro equivalent ones in your 
.sohooJ sv.sLeni. For example, what we have 
la.in-l.ed ^'.superv isor" may have such titles 
as dirooL'or , iielping teacher, coordinator, 
etr. ill your district. 



1 lA. I'lfpposFS FOR FVAliJATION OF TEACHERS 

d8. determ.ines purposes of evaluation 

39. prepares statement of purposes 

Ah*, comniun 1 ca t es purposes to all persons involved 
in evaluation procedures 

A. I . other - • 



MB.. DEVELOPMENT OF CRITERIA FOR EVALUATION 

hi. determines teacher behaviors that are 

impovtant . 

43. determines relative importance of behaviors 

44. determines information to be acquired . . . 

43. deLerriiines procedures to be used for 

acquiring information 

4b. other 



c o 

Qj 

. O O ^ ^ .. 

co'/i' o at. 

Co V A O./'O -r 



IMFOi^M.-VI'i'ON REGARDING TEACHERS 

47. acquires i nt'orination via observation 

48. acqiiires Information directly from tlie 

te-chers being evaluated 

49. ar<iuirc:.s information from indirect sources, 

e.g., students, parents, etc 

50. determines adequacy of information collected 

51. analyzes and Interprets information collected 

52 . oilier . . 



Ot tu. r 

(Spec i I •’ ) 



iiQiiSiiiQ: 0 0 G 0 n 



iiiaiiSiiStO 0 0 idiOiiiQitO 0 0 



iOijOiiip: 0 0 0 :q;;;q:;:0;;:0 o o 
0 0 0 iqiiioiiioiiio 0 o 



0 0 0 ^ 0 

ilillqlii 0 0 0 illililiidpio o o 

isiiiqiliql o o o ianfiiis;;b o o 



;S;:.«:iiq; 0 0 0 iGig.lO ;;p 0 0 

Miliili 0 0 0 o o 



iPiiPiiai 0 0 0 jfliiiSiiSiin 0 0 



iSiiSiiieii 0 0 0 iDiiiKiigii) 0 0 



isyaiifi; 0 0 0 iqiiBiiiGiiio 0 0 

ISHSlibi 0 0 0 o o 

0 0 0 iiqiHbpiljO 0 o 

0 0 0 ll{iijiQiiibillo 0 0 
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Section II: Evaluation of Teachers (continued) 



6 



IID. USE OF INFORMTION 

53. provides written feedback to teachers 

54. provides oral feedback to teachers 

55. provides information to others . . . 

56. maintains record of inforn\ation or of 

information summary 

57. other 



/ / Cu 

^ oV S' 

■S ' . i'/ o\ A 

/fv o ^ ^ other 

V • o ^ c/ ^iSoecifv) 

^ "V ^ ^ ^ ^ ^ 

(jjiifiiia n 0 0 i:ij:iip;iiai 0 u 0 

H-ilill ° ° 0 0 ° ° 

illiiill 0 0 0 liBliiojllll o o o 



SiSiO 0 oiSiiKiO 0 0 

siisiiiiiio o oiisiisiisio o o 



HE. DECISIONS BASED ON EVALUATION 

58. makes decisions regarding changes which should 

be made to improve teacher performance . . . 

59. makes decisions regarding changes of 

assignment 

60. makes recommendations regarding retention . . 

61. makes recommendations regarding promotions 

6L. makes recommendations regarding release . . , 

63. makes recommendations for exceptional salary 

adjustments 

64. other . . 



OiiiKiiKiO 0 0 iiOiiiOiiiei 0 0 0 



PjiiKjiKiO 0 oiipiiSiiSio 0 0 
ill|liSio 0 0 0 0 0 

fiiiii|;0 0 oillpysio 0 0 
fiiiiiilliio 0 oHoiiiKiiKlo o o 



i^ilKlliyio 0 oiiSiii^iiS o o o 



HE. OTHER ACTIVI.'Y (specify) 
65. 



BiiiKiiSio 0 oiiioiiii^iie o 0 0 



66. With relation to the evaluation of teachers, please describe any new procedures 
that you have adopted during the last five yearr. Please enclose any materials 
that are used with these procedures.) 



O 

ERIC 



(Please attach additional comments if this space is not sufficient) 
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Section TI: 



Evaluation of Teachers ^continuO(J) 



OTHER ACTIVITY, coiVt, 

h7. Please describe any revisions which you ant icipate in your procorlurcs for 

evaluation of teachers, (Please enclose any pertinent materials that assist 
in your description,) 



68. With relation to the evaluation of teachers^ please describe what practices 
you use that are most important, effective, or useful. (Please enclose any 
pertinent materials that assist in your description.) 





(Please attach additional comments if this space is not sufficient) 
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SECTION III: 



ADDITIONAL INFORMATION 



8 



1. Have you recently completed (or are you presently engaged in) a research 

or pilot project concerned with either selection or evaluation of teachers? 

Yas If yes, please describe and/or enclose the materials 

pertinent to the project. 

No 



2. Additional comments? 




(Please attach additional comments if this space is not sufficient) 
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APPENDIX 2,2 



Teacher Selection and Evaluation 
Ques t ionnaire 



Dear Respoident: 

This questi -)nnaire has been designed so that you might r^^spond to it 
quickly and so that we might analyze the results and return them to 
you. We are particularly interested in current procedures which you 
have found useful in the selection and evaluation of personnel who 
hold at least a Bachelor of Arts degree. Also, we are interested 
in any research that you have conducted on your selection and evaluation 
procedures , 

Since our sample is relatively small and our project is on a tight 
schedule, it is important to us that the questions be completed 
accurately and promptly. We will appreciate it if you return this 
by February 5 in the enclosed self-addressed envelope. 

Thank you for your consideration and time in completing this 
questionnaire , 



Sincerely , 



Dale L, Bolton 
Principal Investigator 



* * * Vc * Vc * "k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k kkkkkkk k k k k k k k k k k k k k k k k kkkkkk k k k k k 



Name and Title of Respondent 



Organization Name 
Address 

City State Zip Code 



Check the number of employees in your organization: 



100.000 or more 1 10,000 to 15,999 5 

50.000 to 99,999 2 5,000 to 9,999 6 

25.000 to 49,999 3 2,500 to 4,999 7 

16.000 to 24,999 4 less than 2,500 8 



Please return this questionnaire to : 

Selection and Evaluation of Teachers 11-4500 
309 Miller Hall, College of Education 
University of Washington 
Seattle , Washington 98105 



240 



253 



r 



2 



INSTRUCTIONS: Please respond to the following questions. 



1. Does your organization have written policy statements regarding YES 

the selec'"‘^n and/or evaluation of personnel? NO 

2. Does your organization have written procedural statements YES 

regarding the selection and/or evaluation of personnel? NO 



3. If "1^" to both Questions 1 and 2 above, please ignore Questions 
3-10 and answer Questions 11, 12, 13. 

4. If " YES *^ to Questions 1 £T 2 above, we would appreciate 
receiving copies of those statements. (Statements will be 
treated as strictly confidential, and returned or destroyed 
if you wish. ) 



statements attached 
under separate cover 

5. Do the statements in Questions 1 or 2 above describe new or YES 

anticipated procedures in the selection and evaluation of NO 

personnel? 

6. If *'N0" to Question 5, we would appreciate receiving copies 
of statements describing those new or anticipated procedures. 
(Statements will be treated as strictly confidential, and 
returned or destroyed if you wish.) 

statements attached 
under separate cover 

7. Do the statements in Questions 1 or 2 describe anticipated YES 

revisions in the selection and evaluation of personnel? NO 

8. If ^'NO^' to Question 7, we would appreciate receiving copies 
of statements describing those revisions. (Statements will 

be treated as strictly confidential, and returned or destroyed 
if you wish. ) 



statements attached 
under separate cover 



f 



9. 



Do the statements in Questions 1 or 2 indicate which selection 
and evaluation procedures are most beneficial or useful? 



YES 

NO 
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10 . 



I 



If 

below 



wliich 



Ques 

pro 



t 

c 



ion 9, 
edures 



we would 
are most 



appreciate it if you would 
useful or beneficial . 



descr i be 



11 . 



Have you recently completed 
a research project concerned 
tioii of employees? 



(or are you presently engaged in) 
wi th either select ion o r evalua” 



YHS 

NO 



12 . 



If *'YES” to Question 11, 
of statements describing 
if any. (Statements will 
and returned or destroyed 



we would appreciate receiving copies 
the research and pr el iminar^- results 
be treated as strictly contidential 
if you wish. ) 






13. 



statements attached 
under separate cover 



We would appreciate vcur 
procedures which you are 
selection and evaluation 



making any comments regard ing current 
using which might be applicable to 
of teachers. 
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.'PPENDIX 2,2 
Interview Form 



Teacher Selection 
Interview: 



and Evaluation 
Selection 



Person interviewed Organization 

Conducted by 

Time start End Total Date 



1, Describe how criteria for selection are developed (including 
job description, identification of specific behaviors desired, 
informatic/i used to predict,) 

2, Describe how you de;.err.iine employment needs, 

3, DescciLe how you plan and conduct your recruitment program, 

4, What information do you collect regarding all applicants? 



application 

credentials 

tests 



transcripts 

letters of recommend at ion 
t elephone recommendations 



other (explain) 



5, C scribe your interview process: how do you structure the. 

interview, and to whdt extent? Wiat is the sequence of the 
main appraisal interview? Is there any difference in the 
interview of experienced and non--experienced applicants? 
What questions do you find most beneficial in acquiring 
information on applicants? 

6, Describe how you decide: 

a) which applicants are best qualified (rank ordering) 

b) when to ruake an offer (or delay, or reject) 

7, Describe how you check the process of selection to 
determine how successful it is, 

8, Describe Iiow you train the people involved in various 
stages of the selection of applicants , 

9, Describe how your selection procedures have affected 
your organization, 

10. What do you consider the most unusual and/or effective 
aspect of your selection process? 
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If you had relatively unlimited resources, what procedures 
wcniJd vou initiate? 



Interview : Evaluation 

Describe how the purposes of evaluation are determined. 

Describe how criteria for evaluation are developed 
(including behaviors that are deemed important, their 
relative importanc , information to be acquired, and 
procedures to be used for acquiring information, ) 

Describe how, and under what circumstances, information 
regarding performance is gathered (including any prior 
activities that might have occurred.) 

Describe how the information that is collected is analyzed 
and interpreted. (Also describe what records are kept, and 
where , ) 

Hov^ is the information used? (Feedback — how? Information 
to others? Reports? Records?) 

Describe what decisions are made on the basis of the infor- 
mation collected, (Changes to be made: for improvement of 
employee performance; changes of assignment; retention; pro 
motions; release; salary adjustments; other?) 

Describe how you train people involved in various stages of 
the evaluation of teachers , 

Describe how your evaluation procedures have affected 
your organization . 

Wha t do you considtL' the nos t unusual or effective aspect 
of your evaluation process? 

If you had relatively unlimited resources, what procedures 
would you initiate? 



APPENDIX 2,4 



Listing of School Districts to Which 
Questionnaires were Sent 



* Anne Arundel County Public Schools 
Annapolis , Maryland 

Atlanta City Public Schools 
Atlanta, Georgia 

* Aurora School District 
Aurora, Colorado 

Baldwin County Public School 
System 

Bay Minet te , Alabama 

* Baltimore City Public Schools 
Baltimore, Maryland 

Baltimore County Public Schools 
Towson, Maryland 

* Beaverton Public Schools 
Beaverton , Oregon 

* Bellevue Public Schools 
Bellevue, Washington 

* Berkeley City Unified Schools 
Berkeley , California 

** Beverly Hills Unified Schools 
Beverly Hills, California 

* Birmingham City Public Schools 
Birmingham , Michigan 

* Broward County Public School 

System 

Ft, Lauderdale, Florida 

* Camden City Public Schools 
Camden, New Jersey 



* Questionnaire returned 
** Interviewed 



* Chester City Public School 

System 

Chester , Pennsylvania 

Chicago City Public Schools 
Chicago , Illinois 

** Chula Vista City Elementary Schools 
Chula Vista, California 

* Clay County Public School System 
Green Cove Springs, Florida 

* Clayton Public School System 
Clayton, Missouri 

* Cleveland Public School System 
Cleveland , Ohio 

* Cobb County Public Schools 
Marietta, Georgia 

** Columbus Public School System 
Columbus , Ohio 

Conroe Independent School District 
Conroe, Texas 

Cupertino Union Elementary 
Schools 

Cupertino, California 

Dallas ■ iependent School District 
Dallas, Texas 

David Douglas Public Schools /'40 
Portland , Oregon 

* Dearborn City Public Schools 
Dearborn. Michigan 
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DcKalh County Public Schools 
Decatur , (knu'gi a 

Denver l’ul:)lic School System 
Denver , Co 1 orado 

Detroit City Public Schools 
Det ro i t , Michigan 

District of Columbia Public 
Schoo 1 s 

Washington, D. C. 

Duval County Public School System 
.Jacksonvi lie, Florida 

* Englewood School System 
Englewood , Colorado 

Euclid Public School System 
Euclid, Ohio 

** Evanston Public School System 
Evanston, Illinois 

* Fairfax County Public School 

System 

Fairfax , Virginia 

Fulton County Public Schools 
Atlanta, Georgia 

Glendale Unified Public Schools 
Glendale, California 

* Grand Prairie Independent 

School District 
Grant Prairie, Texas 

* Grosse Pointe Public School 

System 

Grosse Pointe, Micliigan 

* Hawaii Public School System 
Honclulu , Hav;aii 

* Highland Park Independent 

School District 
Da 1 las , Texas 



High laud Park Pul)lic Scliool 

District /U07 - 8 and Townslii p 
Iligli School 

Iligliland Park , Illino Ls 

* Highline Public School System 
Seattle, Washington 

* Houston Independent School 

District 
Houston , Tex^is 

* Howard County Public Schools 
Clarksvj lie , Mary] and 

** Indianapolis City Public Schools 
Indianapolis , Indiana 

Jefferson County School District 
Lakewood , Colorado 

** Jefferson Parish Public Scliools 
Gretna , Louisiana 

Ladue Public School System 
Ladue, Missouri 

■’'* Lake Washington Public Schools 
Kirkland , Washington 

Los Angeles Unified Schools 
Los Angeles , California 

** Memphis City Public Schools 
Memphis , Tennessee 

Mesquite Independent School 
District 
Mesqui t e , Texas 

Miami Public School System 
Miami,^ Florida 

** Midland Public Schools 
Midland , Michigan 

Milwaukee Public School System 
Milwaukee , Wisconsin 



O 

ERIC 
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** Minneapolis Special Public 
School System 
Minneapolis , Minnesota 

** Minnetonka Public Schools 
Minnetonka , Minnesota 

* Mobile County Public 

School System 
Mobile, Alabama 

** Montgomery County Public 
Schools 

Rockville , Maryland 

* Nashville Metropolitan Public 

Schools 

Nashville , Tennessee 

Nassau County Public School 
System 

Fernandia Beach, Florida 

National City Elementary 
Schools 

National City, California 

* New Rochelle School System 
New Rochelle, New York 

* New York City Public Schools 
Brooklyn, New York 

Norristown Area Public School 
System 

Norristown , Pennsylvania 

Oakland City Unified Schools 
Oakland , California 

Oak Park Public Schools 
Oak Park, Illinois 

** Orleans Parish Public Schools 
New Orleans, Louisiana 

Palm Beach County Public 
School System 
West Palm Beach, Florida 



* Parma Public School System 
Parma, Ohio 

Pasadena City Unified Schools 
Pasadena, California 

* Pasadena Independent School 

District 
Pasadena, Texas 

* Philadelphia Public School 

System 

Philadelphia , Pennsylvania 

Plaquemines Parish Public Schools 
Pointe a la Hache, Louisiana 

Portland Public Schools 
Portland , Oregon 

Prince Georges County Public 
Schools 

Upper Marlboro , Maryland 

* Racine County Unified School 

District #1 
Racine , Wisconsin 

* Renton School District //403 
Renton , Washington 

** St. Bernard Parish Public Schools 
Chalmet te , Louisiana 

* St . Johns County Public Schoo 1 

System 

S t . Augustine., Florida 

* St . Louis City Public School 

System 

St . Louis , Missouri 

** St . Paul Public Schools 
St. Paul, Minnesota 

* San Diego City Unified Schools 
San Diego, California 



*5V Parkrose Public Schools #3 
Portland, Oregon 
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San Trane i SCO Unified School 
D i s t r i c L 

San 1- ranc.i sco , Cali f ornia 

** Santa Mmiica Unified District 
Scanta Monica, Californici 

* Scarsdale Public Schools 
Scarsdale, New York 

** Seattle i^ibllc Schools 
Seattle, Washington 

* Shaker Heights Public School 

System 

Shaker Heights, Ohio 

** Shelby County Independent 
Sclioo.l District 
Memphis, Tennessee 

* Shoreline Public School System 
Sea tt 1 e, Washington 

* Sliorewood Public School System 
Shorewood, Wisconsin 

** Spring Valley Elementary 
Schools 

La Mesa, California 

* Texas City T.ndependent School 

District 

Texas City, Texas 

University City Public School 
System 

St. Louis, Missouri 

* Upper Arlington Public School 

Sys tem 

Columbus, Ohio 

Warren Township Metropolitan 
School District 
Indianapolis, Indiana 

* Washington Township Schools 
Tndiaru.po Lis , Indiana 

** Waterford Township Public School 
System 

Pontiac , Michigan 



Wauwatosa Public School System 
u\>7 a t o s a , Wisconsin 

* Wayne Township Public School 

System 

Indianapolis , Indiana 

Webster Groves Public School 
System 

Webster Groves , Missouri 

** White Plains Public Schools 
White Plains, New York 

* Whitehall Public School 

System 

Whi tehall , Ohio 

Wor thing ton Public School 
System 

Worthington, Ohio 
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APPENDIX 2.5 



Listing of Businesses to \^ich 
Questionnaires were Sent 



Aluminum Co, of America 
Pittsburgh, Pa. 

American Can Co. 

New York, N. Y. 

* American Cyanamid Co. 

Wayne, N, Y, 

* American Motors Corp. 
Detroit, Mich. 

** A T & T 

New York, N. Y. 

American Tobacco Co. 

New York, N. Y. 

* Anaconda Co. 

New York, N. Y. 

Armour & Co. 

Chicago, 111. 

Avon Products, Inc. 

New York, N. Y. 

** Battelle Memorial Institute 
Columbus, Ohio 

Bendix Corp. 

Detroit , Mich . 

** Boeing Co. 

Seattle, Wash. 

* Boise Cascade Corp. 

Boise, Ida. 

Bristol-Myers Co. 

New York, N. Y. 

Brunswick Corp, 

Chicago, 111, 



Burlington Industries 
Greensboro, N, C, 

Campbell Soup Co . 
Camden, N, J , 

Cannon Mills Co, 
Kannapolis, N, C, 

Caterpillar Tractor Co , 
Peoria, 111, 

Celanese Corp . 

New York, N. Y, 

* Chas. Pfizer & Co, 

New York, N. Y. 

Chevrolet Motor Div. 
Detroit, Mich. 

Chrysler Corp. 

Detroit , Mich . 

Colgate-Palmolive Co . 
New York, N. Y. 

Colt Industries 
New York, N. Y. 

Continental Can Co. 

New York, N. Y. 

Control Data Corp. 
Minneapolis , Minn. 

Crown Cork and Seal Co. 
Philadelphia , Pa . 

* Del Monte Corp. 

San Francisco , Calif . 

Diamond International 
New York, N.. Y. 



* Questionnaire returned 




** Interviewed 
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DonneJioy & Sons 
Cliicngo, III. 


* Hoerner-Waldorf 
St. Paul, Minn. 


'■* Dow Cliemicnl (^o. 
M i ci J and , Mi cli . 


** Honeywell 

Minneapolis, Minn. 


* I'-. I . du Pont de Nemours 

Wilmington, Del. 


Humble Oil and Refining Co 
Houston, Tex. 


F.astman Kodak Co. 
Roc.licster, N. Y. 


IBM 

Armonlc, N. Y. 


Federal-Mogul Corp. 
Detroit, Mich. 


* International Paper Co. 
New York, N. Y. 


Firestone Tire & Rubber 
Akron, Ohio 


>v I T & T 

Nev7 York, N. Y. 


* Fiintkote Co. 

White Plains, N. Y. 


International Utilities 
Toronto, Canada 


* Ford Motors 
Dearborn, Mich. 


Johnson & Johnson 
New Brunswick, N. J. 


General Electric 
New York, N. Y. 


Litton Industries, Inc. 
Beverly Hills 


* General Foods 

White Plains, N. Y. 


Lockheed Aircraft 
Burbank, Calif. 


'■ General Motors Corp. 
Detroit, Mich. 


LTV Aerospace Corp. 
Dallas, Texas 


General Tel. and Elec. 
New York, N. Y. 


Mattel Inc. 
Hawthorne, Calif. 


Genesco Inc. 
.Nashville, Tenn. 


McDonnell Douglas 
St. Louis, Mo. 


Georgia-Pacific Corp. 
Portland, Ore. 


** Minnesota Mining and Mfg. 
St. Paul, Minn. 


Goodyear Tire & Rubber 
Akron, Ohio 


Monsanto 
St. Louis, Mo. 


Gulf & Western Indust. 
New York, N. Y. 


^ National Cash Register 
Dayton, Ohio 


Hart Schaffner & Markx 
Chicago, 111. 


National Dairy Products 
New '•'ork, N. Y. 


Heath Tecna Corp. 
Kent, Wash. 


■' National Industries, Inc. 
Louisville, Ky. 



N. American Rockwell 
El Segundo, Calif. 

* Pacific Car & Foundry 
Renton, Wash. 

PepsiCo, Inc. 

New York, N. Y. 

Polaroid Corp. 

Cambridge, Mass. 

Proctor & Gamble 
Cincinnati , Ohio 

RCA 

New York, N. Y. 

* Ralston Purina Co. 

St. Louis, Mo. 

* Raytheon Co. 

Lexington, Mass. 

* Remington Arms Co. 
Bridgeport, Conn. 

Revlon, Inc. 

New York, N. Y. 

* R. J. Reynolds Tobacco 
Winston-Salem, N. C. 

* Rexall Drug & Chemical 
Los Angeles, Calif. 

** Sears Roebuck 
Chicago, 111. 

* Seattle-First National Bank 
Seattle, Wash. 

** Shell Oil Co. 

New York, N. Y . 

* Singer Co. 

New York, New York 

Sperry Rand Co. 

New York, N, Y. 

Standard Brands Inc . 

New York, N. Y. 



* Standard Oil Co. 

New York, N. Y. 

* State Farm 
Salem, Ore. 

Swift & Co. 

Chicago, 111 . 

TRW Systems 
Redondo Beach, Calif . 

Tecumseh Products Co. 
Tecumseh, Mich. 

** Texas Instruments Inc . 

Dallas , Texas 

Textron Inc. 

Providence, R. I. 

limes Mirror Co. 

Los Angeles, Calif. 

* Union Carbide Corp. 

New York, N. Y. 

United Air craf t 
East Hartford , Conn. 

* U. S. Plywood-Champion Papers 
New York, N. Y. 

* U. S. Steel Corp. 

New York, N. Y. 

W. R. Grace & Co . 

New York, N. Y. 

Warner Lambert Pharmaceutical 
Morris Plains, N. J. 

Weyerhaeuser Co . 

Tacoma, Wash. 

Westinghouse Electric 
Pittsburgh, Pa. 

Xerox Corp. 

Rochester, N. Y. 
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APPENDIX 2.6 



Listing of Governmental Agencies 
to 1^1 ich Questionnaires were Sent 



AEC-NASA Installation 

Space Nuclear Propulsion Office 

Germantown , Md . 

Air Force. Department of the 
The Pentagon 
Washington, D. C. 

Army, Department of the 
The Pentagon 
Washington, D. C. 

** Central Intelligence 
Agency 

Washington , D . C . 

Civil Service Commission, 

U. S. 

Wasliing ton , L. C. 

Electronics Research 
Center 

Cambridge, Mass. 

** Federal Bureau of 
Investigation 
Washington, D. C. 

Jet Propulsion 
Laboratory 
Pasadena, Calif. 

Manned Spacecraft 
Center 

Houston, Texas 

NASA- Ames Research 
Center 

Moffett Field, Calif. 

iLASA-Edwards Flight 
Research Center 
Edwards, Calif. 



* NASA-Goddard Space 

Flight Center 
Greenbelt , M^ • 

* NASA-John F. Kennedy 

Space Center 

Kennedy Space Center, Fla. 

NASA-Langlcy Research 
Center 

Langley Field, Va . 

* NASA-Lewis Research 

Center 

Cleveland , Ohio 

NASA-George C . Marshall 
Space Flight Center 
Huntsville , Alabama 

* NASA-Wallops 

Station 

Wallops , Virginia 

National Aeronautics and 
Space Administration 
Washington , D . C . 

" Navy, Department of the 
The Pentagon 
Washington, D. C. 




* Questionnaire returned 

** Interviewed 
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APPENDIX 3.1 



A Short Discussion of Criterion Variables 



Conditions change under which individuals teach, and this 
affects the behavior that they exhibit. For example, the 
type of supervision in a given building might change 
drastically with a change of principals; consequently more 
or less assistance with certain activities might be 
expected. Also, the behaviors considered important might 
change as the student population changed . The changes in 
these two factors, conditions of work and nature of the 
student population, account for the dynamic nature of the 
teacher behaviors being predicted and the difficulty of 
obtaining long- las ting predictions . Since it is necessary 
to validate t election processes locally and sin ^e the 
criterion variables are dynamic, the validity of predictor 
variables should be reviewed frequently . If we consider 
the two dimensions of differentiation and validity, a 
variable might range from high to low in each dimension. 

If these two dimensions are dichotomized into low and high 
categories , then four conditions emerge : 





Validity 


Dif f erentiation 


1. 


high 


high 


2. 


low 


high 


3. 


low 


low 


4. 


high 


low 




CONCLUSION 



Vaviahles which are non-differentiating or 
lacking in validity may he eliminated from 
consideration in the selection process, 
(position: research staff) 



Condition No. 1 is desirable in that it indicates rele- 
vant variables on which teachers can be measured as 
different. An example of this type of variable might 
be creativity which is observed in practice teaching 
activities. Condition No. 2, however, is meaningless for 
validating a selection procedure. For example, height 
might differentiate among teachers, but is probably 
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irrelevant: to most scliool system goals. It seems obvious 
that variables identified by Condition No. 3 are to be 
rejected. liowever , Vciriables represented by Condition 
No . 4 are less easily disposed of . For example , observa- 
tions of kindergarten teachers’ empathy for children 
during practice teaching might be considered very rele- 
vant (valid) for the job, yet a group of applicants for 
kindergarten positions may vary only slightly on this 
characteristic. Therefore, this factor becomes useless 
for validating the selection process. In fact, if the 
measurement of all kindergarten applicants on this variable 
yields similar results, then the correlation with any 
other variables will approach zero and there will be no 
predictive value. 

The selection of variables might proceed by first identi- 
fying variables which have evidence of being valid, check- 
ing their discrimination, and finally checking the validity 
of those which do discriminate. 

\fhere multiple predictor variables are used to predict 
criteria, one should ascertain whether they interact with 
each other so that one compensates for the other. For 
example , does intelligence compensate for hard work or 
diligence cr vice versa; would experience substitute for 
education? 



Some pvediotov variables interact with each 
other in such a manner that one compensates 
for the other; where this occurs^ iyidividuals 
assessed on a number of dimensions may differ 
quantitatively on several dimensions and still 
ho.ve an overfall prediction of similar utility to 
the organization. (position: research staff) 



A complicating factor to the interaction of variables is 
the fact that compensation may occur only within certain 
ranges of the variable. For example, hard work may com- 
pensate for intelligence if a minimum level of intelli- 
gence is available. This is illustrated in the figure 
below where the shaded region is acceptable predicted 
behavior. Note that no behavior is satisfactory if 
variable ^ is slightly larger than x^ or if variable ^ 
falls below y^. However, if variable ^ is slightly 
larger than x^, a large amount of variable _b will compen- 
sate for the small amount of a . It seems obvious that 
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Figure A3.1. Compensatory Nature of Variables. 




the use of multiple cutoff scores would now allow 
compensation of some abilities for others required on 
the job. Since assumptions regarding compensation and 
the use of multiple cutoffs do yield different results, 
it is essential to specify where compensation occurs 
if consistent decisions are to be made. 

In addition, such compensation may occur in some 
positions and not in others. Therefore, it is necessary 
to decide which variables interact for each position 
that is considered to be unique. 
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